Computer-implemented method, wearable device, computer program and computer readable medium for assisting the movement of a visually impaired user

ABSTRACT

S3—Determining, repeatedly updating and storing, of at least one navigation path together with associated navigation guiding instructions for the visually impaired user to navigate from the current position of the visually impaired user to a point of interest, repeatedly selecting one preferred navigation path from the at least one navigation path, and repeatedly sending to the visually impaired user the preferred navigation path, together with associated navigation guiding instructions.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims prior to European Pat. App. No. 21154462, filedJan. 29, 2021, which is incorporated herein by reference.

FIELD

The invention is related to the field of assistance of the movement ofvisually impaired persons. In particular the invention is related to amethod and a wearable device for assisting the movement of a visuallyimpaired person, which throughout the invention shall be called visuallyimpaired user. The term “visually impaired” shall encompass throughoutthe invention the moderate impairment as well as severe impairment(blindness).

BACKGROUND

Studies have shown that 36 million people were affected by severeimpairment in 2015 While 216.6 million had moderate to severe visualimpairment. While these numbers are increasing, people become more awareof their needs and solutions targeted at aiding visual disabilityemerge. For example, cities are becoming more accessible to blindindividuals using classic navigation methods such as the walking cane orthe guide dog. Technological solutions also emerge and begin beingaccepted by the blind and visually impaired community. Solutions such asthe ones proposed by OrCam Inc. or Microsoft Inc. have seen variouslevels of adoption. However, despite the advancements in technology, themost used solution for the visually impaired persons is still thewalking cane.

Technological solutions for assisting the visually impaired persons arenot new. Early efforts can be attributed to Paul Bach-Y-Ritta et al. inU.S. Pat. No. 3,594,823A. There, visual signal has been translated tohaptic feedback on the back of blind individuals.

Generally speaking, the technological solutions for assisting thevisually impaired persons emerged in one or more of the followingcategories:

-   -   Sensorial substitution—using an available sense to represent        information not normally received through said sense. Several        other efforts have been made in the following years to replace        visual information using other senses, such as the tactile        sense, or sound,    -   Ways of generating paths for the visually impaired person and        communicating said paths to him/her, by using the sensorial        substitution,    -   Ways of localizing the position of the visually impaired person        in its environment and/or localizing various objects that can be        considered obstacles and/or target destinations,    -   Ways of communicating the information regarding the environment,        paths, obstacles to the visually impaired person and receiving        feedback from him/her.

Various solutions address one or more of the above-captioned categories.

U.S. Pat. No. 9,915,545 published 26 Jul. 2015 discloses a method forproviding directions to a blind person using a smart device. The methodincludes detecting, by at least two sensors and in response to aselection of a find mode of the smart device, image data correspondingto a surrounding environment of the smart device and positioning datacorresponding to a positioning of the smart device; receiving by aninput device the desired object or desired location; determining by aprocessor the initial location of the smart device based on the imagedata, the positioning data and map data stored in a memory of the smartdevice; providing by the output device of directions to the desiredobject based on the initial location of the smart device and the mapdata. U.S. Pat. No. 9,629,774 published 16 Jul. 2015 discloses a smartnecklace that includes a body defining at least one cavity and having aneck portion and first and a second side portions. The necklace includesa pair of stereo cameras that is configured to detect image dataincluding depth information corresponding to a surrounding environmentof the smart necklace. The necklace further includes a positioningsensor configured to detect positioning data corresponding to apositioning of the smart necklace. The necklace includes anon-transitory memory positioned in the at least one cavity andconfigured to store map data and object data. The smart necklace alsoincludes a processor positioned in the at least one cavity, coupled tothe pair of stereo cameras, the positioning sensor and thenon-transitory memory. The processor is configured to determine outputdata based on the image data, the positioning data, the map data and theobject data.

In an academic article published 15 Nov. 2016, “Wearable IndoorNavigation System with Context Based Decision Making for VisuallyImpaired”, the authors Xiaochen Zhang et al. present a wearable indoornavigation system for the visually impaired. The system usesSimultaneous Localisation and Mapping SLAM and semantic path planningfor the localisation and navigation, integrating m sensors and feedbackdevices such as an RGB-D camera, an Inertial Measurement Unit IMU and aweb camera. The system applies the RGB-D based visual odometry algorithmto estimate the user's location and orientation and the InertialMeasurement Unit IMU to refine the orientation error. Major landmarkssuch as room numbers and corridor corners are detected by the web cameraand the RGB-D camera and matched to the digitalized floor map so as tolocalize the user. The path and motion guidance are generated to guidethe user to a desired destination. The article suggests a way to improvethe fitting between the rigid commands and optimal machine decisions forhuman beings in a context-based decision-making mechanism on pathplanning to resolve user's confusions caused by incorrect observations.

Disadvantages of Prior Art

In respect to the sensorial substitution, the known solutions generallypropose an improper replacement of the bandwidth of eyesight which isknown to be far greater than the one embedded in the auditory and/or thehaptic senses, providing either too scarce information or too muchnon-essential information, confusing or annoying the user.

In respect to the ways of generating the map as well as the ways ofgenerating paths for the visually impaired person and communicating saidpaths to him/her, the Navigation/GPS-based methods of prior art offer ageneral path but do nothing to avoid obstacles or living beings.

In respect to the ways of localizing the position of the visuallyimpaired person in its environment and/or localizing various objectsthat can be considered obstacles and/or target destinations, methodssuch as Microsoft Seeing AI or OrCam recognize only the objects that arepresent in their limited field of view but do not offer sufficientinformation about how to reach the target destinations. In general, theknown solutions have no or reduced possibility to store the position ofthe objects detected by the sensors and, consequently no or reducedpossibilities to recognize objects detected in the past. The knownsolutions have no possibilities to provide full information about theobjects, information that may be of use for the visually impaired personsuch as: if a chair is occupied by other person, the sizes and otherphysical and chemical characteristics of the objects which leads to theneed that the visually impaired person touches said objects in order toperceive said physical and chemical characteristics, which is not veryhygienic, can be dangerous and may take too much time.

In respect to the ways of communicating the information regarding theenvironment, paths or obstacles to the visually impaired person andreceiving feedback from him/her, most of current solutions communicatethe information too slowly to the visually impaired person and/orreceive from him/her the feedback too slow, which leads to difficult anddelayed modification of the initial path.

For example, in an embodiment of U.S. Pat. No. 9,629,774, although thesmart necklace may recognize stairs, exits restrooms or empty seats,said necklace is not configured to provide more in-depth informationsuch as characteristics of the objects: orientation, where is the latchor doorknob or if the empty seat is dirty.

U.S. Pat. No. 9,629,774 teaches a limited number of types of commandsthat the smart necklace can transmit to the visually impaired user:different degrees of turns, such as 45—degree turn, a 90-degree turn,left turn, right turn. Using this way of communicating the information,U.S. Pat. No. 9,629,774 is rigid when compared with the natural pathgeneration of the non-visually impaired person.

U.S. Pat. No. 9,629,774 teaches about a map that contains only locationof various objects without any other characteristics.

U.S. Pat. No. 9,915,545 teaches a method for providing directions to ablind user of an electronic device, the directions being from a currentlocation of the electronic device to a location of a desired object. Inthis method there is no differentiation between the types of areas onwhich the paths are generated, there are no relationships built betweenvarious objects, some important of the characteristics of the objectsand living beings are left out, such as the detection of the emotionalstatus of the living beings or the degree of cleanliness of a surface,and the content of the map is reduced to the information from the fieldof view of the sensors.

Problem Solved by the Invention

The problem to be solved by the invention is to provide for a method forassisting the movement of the visually impaired user that allows theuser to navigate indoor and outdoor in a manner closer to the navigationof a non-visually impaired user. In particular, the problem to be solvedby the invention is:

-   -   to provide more accurate and more detailed representation of the        environment of the user including providing more accurate and        more detailed relationships between the objects and living        beings with the objective of a safer navigation of the visually        impaired user and a more concrete navigation goal;    -   to provide more accurate and more detailed generation of the        navigation paths;    -   to provide better guidance of the visually impaired user along        the navigation paths in terms of accuracy and security;    -   to provide better guidance of the visually impaired user along        the navigation paths in terms of more comfort and giving the        possibility to take decisions similar to those of a non-visually        impaired user;

SUMMARY OF THE INVENTION

In order to solve the problem, the inventors conceived in a first aspectof the invention a method for assisting the movement of a visuallyimpaired user by means of a wearable device, comprising the followingsteps:

S1

Acquiring data from the environment of the visually impaired user bymeans of a sensory unit 2 of the wearable device 1, sensing from a fieldof view 20, sending said acquired data to a Sensory Fusion sub-unit 30of a Processing and control unit 3 of the wearable device 1,

S2

Fusing the acquired data by the Sensory fusion sub-unit 30, sending thefused data to a Live Map sub-unit 31 of the Processing and control unit3,

Creating, repeatedly updating and storing by the Live Map sub-unit 31 ofa Live Map 310, the Live Map 310 comprising the following data:

1. Live Map determinations based on fused data received from the SensoryFusion sub-unit 30, including:

-   -   A position and an orientation of the sensory unit 2,    -   A plurality of objects On,    -   A plurality of living beings (Ln),        2. Live Map determinations that are generated based on a        plurality of relationships Rn between the plurality of objects        On and/or the plurality of living beings Ln received from a        Relationship Manager sub-unit 32 of the Processing and control        unit 3,        3. Live Map determinations that are generated based on a free        area A defined as an ensemble of areas on a ground not occupied        by the plurality of objects On and the plurality of living        beings Ln, the free area A including:    -   A walkable area WA that satisfies a set of permanent        predetermined walkable area requirements,    -   A conditional walkable area CWA that satisfies said set of        permanent predetermined walkable area requirements, and at least        one predictable conditional walkable area requirement,        S3

Automatically or in response to a request from the visually impaireduser determining, by a Navigation Manager sub-unit 33 of the Processingand control unit 3, repeatedly updating and storing, of at least onenavigation path Pn and associated navigation guiding instructions forthe visually impaired user to navigate from the current position of thesensory unit 2 to a point of interest PI selected among the plurality ofobjects (On) and/or the plurality of living beings Ln, Automatically orin response to a request from the visually impaired user repeatedlyselecting a preferred navigation path SP from the at least onenavigation path Pn, that satisfies at least two navigation pathrequirements:

i) Passes through the walkable area WA and/or on the conditionalwalkable area CWA, and

ii) Meets a set of safety requirements including a non-collisionrequirement, and a non-aggressivity requirement.

Wherein any request from the visually impaired user is made by usinghaptic means 51 or audio means 52 of a User commands interface 5, saidrequests being received by the Navigation Manager sub-unit 33 via a Usercommands interface Manager sub-unit 34 of the Processing and controlunit 3,

and transmitting by the Navigation Manager sub-unit (33) to a FeedbackManager sub-unit 35 of the Processing and control unit (3)

i) of the preferred navigation path SP and

ii) of the associated navigation guiding instructions,

wherein, when the preferred navigation path SP passes through theconditional walkable area CWA, the Navigation Manager sub-unit) sends tothe Feedback Manager sub-unit the associated navigation guidinginstruction corresponding to said at least one predictable conditionalwalkable area requirementS4

Providing, by the Feedback Manager sub-unit 35 guidance to the visuallyimpaired user, along the preferred navigation path SP, by using guidingmodes for transmitting each associated navigation guiding instruction,each navigation instruction comprising haptic and/or auditory cues sentby the Feedback Manager sub-unit 35 to a Feedback unit 4 of theProcessing and control unit (3), said Feedback unit 4 comprising:

-   -   haptic feedback actuators 41 configured for placement on the        head of the visually impaired user, and/or    -   auditory feedback actuators 42 configured for placement to one        or both to the ears of the visually impaired user,        wherein the guiding modes for each associated navigation guiding        instruction are selected by the visually impaired user by the        User commands interface 5 and through user commands that are        received by the Feedback Manager sub-unit 35 via the User        commands interface Manager sub-unit 34.

In a second aspect of the invention, it is provided a wearable device 1for assisting the movement of a visually impaired user, comprising:

-   -   a Sensory unit 2 configured to be placed on the head of the        visually impaired user, comprising basic sensors:    -   a Camera 21,    -   a Depth sensor 22,    -   an Inertial Measurement unit 23    -   a Sound localisation sensor 24    -   a Processing and control unit 3 comprising:    -   a Sensory fusion sub-unit 30 comprising:    -   a Localisation module 301,    -   a Walkable Area Detection module 302,    -   an Orientation Computation module 303,    -   a Sound Direction Localisation module 304,    -   a Sound Classification module 305,    -   an Object 2D Characteristics Extraction module 306,    -   an Object 3D Characteristics Fusion module 307,    -   an Object Sound Characteristics Fusion module 308,    -   a Live Map sub-unit 31,    -   a Relationship Manager sub-unit 32,    -   a Navigation Manager sub-unit 33,    -   a User commands interface Manager sub-unit 34,    -   a Feedback Manager sub-unit 35,    -   a Sound representation sub-unit 36,    -   a Feedback unit 4 configured to be placed on the head of the        visually impaired user, comprising:    -   a plurality of haptic feedback actuators 41 comprising:    -   left haptic feedback actuators 411,    -   right haptic feedback actuators 412,    -   centre haptic feedback actuators 413,    -   a plurality of auditory feedback actuators 42 comprising:    -   left auditory feedback actuators 421,    -   right auditory feedback actuators 422,    -   a User commands interface 5 configured to be placed on the head        of the visually impaired user    -   comprising:    -   a plurality of user commands haptic means 52,    -   a plurality of user commands audio means 52,    -   a Power storage unit 6,    -   a memory M, and        electronic communications means between the Sensory unit 2, the        Processing and control unit 3, the Feedback unit 4, the User        commands interface 5, the Power storage unit 6 and the memory M,        by communication protocols,    -   wherein the wearable device is configured to apply the steps of        the method according to any embodiment of the invention.

In a third aspect of the invention, it is provided a computer programcomprising instructions which, when the program is executed by thewearable device causes the wearable device 1 to carry out the steps ofthe computer-implemented method for assisting the movement of a visuallyimpaired user, in ny of the preferred embodiments, includingcombinations thereof.

In a fourth aspect of the invention, it is provided a computer readablemedium having stored thereon instructions which, when executed by thewearable device 1, causes the wearable device 1 to carry out the stepsof the computer-implemented method, in any of the preferred embodiments,including combinations thereof.

In a fifth aspect of the invention, it is provided a non-transitorycomputer-readable storage device storing software comprisinginstructions executable by one or more computers which, upon suchexecution, cause the one or more computers to perform operations of thecomputer-implemented method, in any of the preferred embodiments,including combinations thereof

In a sixth aspect of the invention, it is provided a system comprisingone or more computers and one or more storage devices storinginstructions that are operable, when executed by the one or morecomputers, to cause the one or more computers to perform operations ofthe computer-implemented method, in any of the preferred embodiments,including combinations thereof. According to one example implementation,a computer-implemented method comprising includes acquiring data from anenvironment of a visually impaired user, comprising a sensory unit of awearable device sensing from a field of view, sending the acquired datato a sensory fusion sub-unit of a processing and control unit of thewearable device, fusing the acquired data by the sensory fusionsub-unit, sending the fused data to a live map sub-unit of theprocessing and control unit, and creating, repeatedly updating, andstoring, by the live map sub-unit, a live map. The live map includes oneor more live map determinations that are generated based on the fuseddata received at the processing and control unit from the sensory fusionsub-unit, including a position and an orientation of the sensory unit, aplurality of objects, and a plurality of living beings, one or more livemap determinations that are generated based on a plurality ofrelationships between the plurality of objects or the plurality ofliving beings or between the plurality of objects and the plurality ofliving beings that are received from a relationship manager sub-unit ofthe processing and control unit, one or more live map determinationsthat are generated based on a free area that is defined as an ensembleof areas on a ground not occupied by the plurality of objects and theplurality of living beings, the free area including a walkable area thatsatisfies a set of permanent predetermined walkable area requirements,and a conditional walkable area that satisfies the set of permanentpredetermined walkable area requirements, and at least one predictableconditional walkable area requirement. The method includes automaticallyor in response to a first request from the visually impaired user,determining, by a navigation manager sub-unit of the processing andcontrol unit, repeatedly updating and storing, at least one navigationpath and associated navigation guiding instructions for the visuallyimpaired user to navigate from a current position of the sensory unit toa point of interest selected among the plurality of objects or theplurality of living beings or the plurality of objects and the pluralityof living beings, automatically or in response to a second request fromthe visually impaired user, repeatedly selecting a preferred navigationpath from the at least one navigation path that (i) passes through thewalkable area or on the conditional walkable area or on the walkablearea and on the conditional walkable area, and (ii) meets a set ofsafety requirements including a non-collision requirement, and anon-aggressivity requirement, where any request from the visuallyimpaired user is made by using haptic means or audio means of a usercommands interface the requests being received by the navigation managersub-unit via a user commands interface manager sub-unit of theprocessing and control unit, transmitting, by the navigation managersub-unit to a feedback manager sub-unit of the processing and controlunit, the preferred navigation path and the associated navigationguiding instructions, where, when the preferred navigation path passesthrough the conditional walkable area, the navigation manager sub-unitsends to the feedback manager sub-unit the associated navigation guidinginstruction corresponding to the at least one predictable conditionalwalkable area requirement, providing, by the feedback manager sub-unit,guidance to the visually impaired user, along the preferred navigationpath, using guiding modes for transmitting each associated navigationguiding instruction, each navigation instruction comprising haptic orauditory cues sent by the feedback manager sub-unit to a feedback unitof the processing and control unit, the feedback unit including hapticfeedback actuators configured for placement on the head of the visuallyimpaired user, or auditory feedback actuators configured for placementto one or both ears of the visually impaired user, or haptic feedbackactuators configured for placement on the head of the visually impaireduser and auditory feedback actuators configured for placement to one orboth ears of the visually impaired user, where the guiding modes foreach associated navigation guiding instruction are selected by thevisually impaired user by the user commands interface and through usercommands that are received by the feedback manager sub-unit via the usercommands interface manager sub-unit.

Some implementations may include one or more of the following features.For example, the method may include creating and updating the live map,including repeatedly determining the position and orientation of thesensory unit, a position, orientation and characteristics of theplurality of objects and of the plurality of living beings, based on thefused data received from the sensory fusion sub-unit, and repeatedlysending the created and updated live map to a localization module of thesensory fusion sub-unit, repeatedly generating and updating, by therelationship manager sub-unit, the plurality of relationships betweenthe plurality of objects or the plurality of living beings or theplurality of objects and the plurality of living beings based on thedata acquired from the live map including applying a set of thepredetermined relations requirements, and repeatedly sending the updatedplurality of relationships to the live map, repeatedly localizing, by alocalization module the position and orientation of the sensory unitwith respect to the plurality of the objects, and, to the plurality ofliving beings of the live map using localization algorithms applied tothe data received from the sensory unit and data from the of the livemap and repeatedly sending the localization data of the position andorientation of the sensory unit to a walkable area detection module ofthe sensory fusion sub-unit, repeatedly determining, by the walkablearea detection module, the free area based on the data received from thesensory unit, the data received from the localization module, the set ofpermanent predetermined walkable area requirements, and the at least onepredictable conditional walkable area requirement calculated and storedin the memory, and repeatedly sending the updated free area to the livemap, and repeatedly storing the updated live map in the memory. The livemap may be updated by the sensory fusion sub-unit using simultaneouslocalization and mapping (SLAM) algorithms. The method may includesending an information request by the visually impaired user to a soundrepresentation sub-unit of the processing and control unit regarding atleast one object selected from the plurality of objects or at least oneliving being selected from the plurality of living beings, extracting bya sound representation sub-unit of the processing and control unit fromthe live map the information regarding the selected at least oneparticular object or at least one particular living being; representingthe extracted information as corresponding spatialized sounds,transmitting the spatialized sounds to the visually impaired user by thefeedback unit, selecting, by the visually impaired user of the point ofinterest from the plurality of objects or from the plurality of livingbeings, and transmitting the corresponding selection request to thenavigation manager sub-unit. The method may include determining by thenavigation manager wandering path together with the associatednavigation guiding instructions for the visually impaired user, andsending the wandering path and the associated navigation guidinginstructions to the feedback manager sub-unit. The haptic cues may varyin duration, periodicity, intensity or frequency of the vibrationaccording to predetermined preferred navigation path complexitycriteria, and the audio cues may vary in frequencies, duration,repetition intensity, or 3d spatial virtualization according to thepredetermined preferred navigation path complexity criteria. Athree-dimensional walkable tunnel may be defined as a virtual tunnel ofpredetermined cross-section, having as horizontal longitudinal axis thepreferred navigation path, and wherein the guiding mode furthercomprises specific haptic cues sent to the visually impaired user whenthe visually impaired user is approaching the virtual walls of thewalkable tunnel. The preferred navigation path may be divided intopredetermined segments delimited by a plurality of milestones, and theguiding mode may include haptic cues or auditory cues signaling theposition of a next at least one milestone providing associatednavigation guiding instructions to the visually impaired user from acurrent milestone to a subsequent milestone, and the length of thepredetermined segments may vary depending on the complexity and lengthof the preferred navigation path. The guiding mode may include hapticcues or auditory cues or haptic and auditory cues signaling a directionon the preferred navigation path. The direction on the preferrednavigation path may be determined by the line defined by an origin ofthe sensory unit and an intersection of the preferred navigation pathwith a circle having an origin at the position of the sensory unit and aradius with a predetermined length, and the auditory cues signaling thedirection on the preferred navigation path may originate from aspatialized sound source placed at a predetermined first distance of thespatialized sound source s with respect to the sensory unit. Theauditory cues may be spatialized sounds originating from a spatializedsound source that virtually travels along a predetermined seconddistance on the preferred navigation path from the position of thesensory unit until the spatialized sound source reaches the end of thepredetermined second distance and back to the position of the sensoryunit.

In another general implementation, a wearable device for assisting themovement of a visually impaired user includes a sensory unit configuredto be placed on the head of the visually impaired user, including acamera, a depth sensor, an inertial measurement unit, and a soundlocalization sensor. The device includes a processing and control unitincluding a sensory fusion sub-unit including a localization module, awalkable area detection module, an orientation computation module, asound direction localization module, a sound classification module, anobject 2d characteristics extraction module, an object 3dcharacteristics fusion module, and an object sound characteristicsfusion module. The device includes a live map sub-unit, a relationshipmanager sub-unit, a navigation manager sub-unit, a user commandsinterface manager sub-unit, a feedback manager sub-unit, and a soundrepresentation sub-unit. The device includes a feedback unit configuredto be placed on the head of the visually impaired user, including aplurality of haptic feedback actuators including left haptic feedbackactuators, right haptic feedback actuators, center haptic feedbackactuators, a plurality of auditory feedback actuators including leftauditory feedback actuators, and right auditory feedback actuators Thedevice includes a user commands interface configured to be placed on thehead of the visually impaired user including a plurality of usercommands haptic means, and a plurality of user commands audio means. Thedevice includes a power storage unit, a memory, and electroniccommunications component between the sensory unit, the processing andcontrol unit, the feedback unit, the user commands interface, the powerstorage unit and the memory.

Example implementations may include one or more of the followingfeatures. The sensory unit may include at least one additional sensor,from among a global positioning sensor configured to determine theabsolute position of the sensory unit, or a temperature sensorconfigured to determine the temperature of the objects and of the livingbeings. The sensory fusion sub-unit may include a relative-to-absoluteconversion module that is configured to fuse the data from the objectsound characteristics fusion module with the data regarding the absoluteposition of the sensory unit, and an object temperature characteristicsfusion module that is configured to fuse the data from the object soundcharacteristics fusion module with the data regarding the temperature ofthe objects and of the living beings, and to send the fused data to thelive map sub-unit.

Other example implementations may include a system including one or moreprocessors; and one or more non-transitory machine-readable storagedevices storing instructions that are executable by the one or moreprocessors to perform operations corresponding to the disclosed methods,or a non-transitory computer storage medium encoded with a computerprogram, the computer program comprising instructions that when executedby one or more processors cause the one or more processors to performoperations corresponding to the disclosed methods.

Any feature in one aspect of the invention may be applied to otheraspects of the invention, in any appropriate combination. In particular,method features may be applied to device features, and vice versa.

Wherever applicable, means—plus—function features may be expressedalternatively in terms of their corresponding structure, such as asuitably programmed processor and associated memory. Particularcombinations of the various features of the invention can be implementedand/or supplied and/or used independently.

Advantages of the Invention

The main advantages of this invention are the following:

-   -   It provides a safer navigation of the visually impaired user and        a more accurate access to the objects and living beings of        everyday life due to:    -   a more accurate and more detailed representation of the        environment of the user;    -   a more accurate and more detailed generation of the navigation        paths;    -   a more accurate and a safer guidance of the user along the        navigation paths;    -   It provides a navigation experience closer to the experience of        non-visually impaired user in terms of comfort and possibility        to make decisions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 Schematic representation of the method and of the wearable device1 according to the invention

FIG. 2A 1 and FIG. 2A 2 are representations of two component wearabledevice

FIG. 2B Single component wearable device

FIG. 3A and FIG. 3B Schematic representation of the field of view ofbasic sensors 20

FIG. 4 Schematic representation of the content of the reality and of theLive Map 310

FIG. 5 Schematic representation of the creation of the Live Map 310 whenthe wearable device 1 uses only the basic sensors

FIG. 6 Trimetric view of one of the preferred embodiments, showcasingthe headset component 11 as viewed from the back of the head of thevisually impaired user and detailing the positioning of the componentsof the Feedback unit 4 and of the components of the User commandsinterface 5

FIG. 7 Frontal view of a detail of the preferred embodiment of FIG. 6,showcasing the headset component 11 as viewed from the forehead of thevisually impaired user and detailing the positioning of the four basicsensors and detailing the positioning of the components of the Usercommands interface 5

FIG. 8 A Schematic representation of the guiding modes using thewalkable tunnel T

FIG. 8 B Trimetric view of the embodiment showing guiding mode by usingthe walkable tunnel T

FIG. 9 Trimetric view of the embodiment showing guiding mode by usingmilestones 93

FIG. 10 Schematic representation of the embodiment showing guiding modeby using the walking tunnel T and the milestones 93

FIG. 11A Schematic representation of the embodiment showing guiding modeby using haptic cues

FIG. 11B Schematic representation of the embodiment showing guiding modeby using auditory cues

FIG. 12 Schematic representation of the embodiment showing guiding modeby using spatialized sounds originating from the spatialized soundsource S that virtually travels along the preferred navigation path SP

FIG. 13 Schematic representations of the embodiment corresponding to thewearable device 1 that comprises two additional sensors 25 and 26.

FIGS. 14 to 17 Exemplification of the method—Example No. 1

FIG. 14 Top view of the real-life scenario scene

FIG. 15. Trimetric view of the real-life scenario scene

FIG. 16. Detail from the secondary path 912

FIG. 17. Detail—the doorbell 841 replaces the door 84 as point ofinterest PI

FIGS. 18 to 28 Exemplification of the method—Example No. 2. The window85 is represented symbolically

FIG. 18 Example No. 2-1—schematic representation of the step S.3-0.2 ofthe method wherein the spatialized sound S86 is perceived in the samelocation of the window 85

FIG. 19A Example No. 2-2—schematic representation of the step S.3-0.2 ofthe method wherein the spatialized sound S86 is an exaggerated or“zoomed” representation in respect to the window 85 on the elevationspatial dimension

FIG. 19B Example No. 2-3—schematic representation of the step S.3-0.2 ofthe method wherein the spatialized sound S86 is an exaggerated or“zoomed” representation in respect to the window 85 as the potentialpoint of interest PPI on the azimuth spatial dimension.

FIG. 20 Example No. 2-4—schematic representation of the step S.3-0.2 ofthe method wherein the two windows 85-1 and 85-2, placed at non-equaldistances from the visually impaired user, are sound-represented withdifferent frequency characteristic of the virtualized sounds S86 f-1,and, respectively, 586 f-2 depending on the distances of the windows85-1, and 85-2 from the visually impaired user.

FIG. 21 Example No. 2-5—schematic representation of the step S.3-0.2 ofthe method wherein the margins of the window 85 are sound-represented bytwo spatialized sounds S86P-E1, and S86P-E2 having different encodingcharacteristics depending on the distance of each the extremitiesrelative to the visually impaired user.

FIG. 22 Example No. 2-6—schematic representation of the step S.3-0.2 ofthe method showing the representation of the dimensions S86P, and S86Lof the window 85

FIG. 23 Example No. 2-7—schematic representation of the step S.3-0.2 ofthe method wherein the shape of the window 85 is represented by twospatialized punctiform sound sources S86P1 and S86P2

FIG. 24 Example No. 2-8—schematic representation of the step S.3-0.2 ofthe method wherein the shape of the window 85 is represented by twospatialized sound sources S86 f 1, and S86 f 2 having differentfrequency on the azimuth.

FIG. 25 Example No. 2-9—schematic representation of the step S.3-0.2 ofthe method wherein the shape of the window 25 is represented by thesingle spatialized sound source S86 that virtually travels from thestarting point starting point t₀ on the contour of the window 85 untilit reaches back the starting point starting point t₀ .

FIG. 26 Example No. 2-10—schematic representation of the step S.3-0.2 ofthe method wherein the shape of the window 25 is represented by thesingle spatialized sound source S86 that virtually travels in an angledpattern.

FIG. 27 Example No. 2-11—schematic representation of the step S.3-0.2 ofthe method showing the representation of two windows 85-1, and 85-2,placed at different distances relative to the visually impaired user.

FIG. 28 Example No. 2-12—schematic representation of the step S.3-0.2 ofthe method showing the sound representation of a window 85 by twospatialized sounds S861, S862 that start at the same time, and travel inangled patterns inside the interior frame of the window 85.

LIST OF REFERENCES

This list includes references to components, parameters or criteriapresents in the description and/or drawings. It is created to ease thereading of the invention.

-   1 Wearable device-   11 Headset component-   12 Belt-worn component-   12 Wrist component—not represented graphically-   13 Hand-held component—not represented graphically-   2 Sensory unit-   Basic sensors: 21, 22, 23,24-   20 field of view of the basic sensors-   21 Camera-   22 Depth sensor-   21-22 Camera and Depth Sensor—not represent graphically-   23 Inertial Measurement unit-   24 Sound localisation sensor-   21-22 Camera and Depth Sensor—not represent graphically-   Additional sensors 25,26-   25 Global positioning sensor-   26 Temperature sensor-   3 Processing and control unit-   30 Sensory Fusion sub-unit-   301 Localisation module-   302 Walkable Area Detection module-   303 Orientation Computation module-   304 Sound Direction Localisation module-   305 Sound Classification module-   306 Object 2D Characteristics Extraction module-   307 Object 3D Characteristics Fusion module-   308 Object Sound Characteristics Fusion module-   309-1 Relative to Absolute Conversion module-   309-2 Object Temperature Characteristics fusion module-   31 Live Map sub-unit-   310 Live Map-   32 Relationship Manager sub-unit-   33 Navigation Manager sub-unit-   34 User commands interface Manager sub-unit-   35 Feedback Manager sub-unit-   36 Sound representation sub-unit-   4 Feedback unit-   41 Haptic feedback actuators-   411 Left feedback actuators-   412 Right feedback actuators-   413 Centre feedback actuators-   42 Auditory feedback actuators-   421 left auditory feedback actuators-   422 right auditory feedback actuators-   5 User commands interface-   51 User commands haptic means-   52 User commands audio means-   6 Power storage unit—not represented graphically-   M Memory—not represented graphically-   7 Communication unit—not represented graphically-   Content of the Live Map 310    Live Map Determinations Based on the Fused Data Received from the    Sensory Fusion Sub-Unit 30    -   position and orientation of the sensory unit 2    -   On a plurality of objects    -   Ln a plurality of living beings—not represent graphically        Live Map Determinations Based on the Data Received from the        Relationship Manager Sub-Unit 32:    -   a plurality of relationships Rn between the plurality of objects        On and/or the plurality of living beings Ln=relations—not        represent graphically        A Free Area A:    -   WA walkable area    -   CWA conditional walkable area    -   NA non-walkable area        Minimum Requirements    -   For the areas:    -   a set of permanent predetermined walkable area requirements    -   at least one predictable conditional walkable area requirement    -   For the relations:    -   categories of predetermined relations requirements including:    -   predetermined parent-child relations and    -   predetermined conditional relations    -   For the navigation paths Pn:    -   two navigation path requirements    -   safety requirements    -   a non-collision requirement    -   a non-aggressivity requirement    -   be on the WA or on the CWA        Criteria for Selection    -   a set of path selection criteria    -   cost criteria    -   cost-time to destination criteria    -   comfort criteria    -   predetermined preferred navigation path complexity criteria        Criteria for the Spatialized Sounds    -   predetermined spatialized sounds criteria        Requests and Selections by the Visually Impaired User    -   an initiation request    -   a selection request    -   an information request        Paths Determined by the Navigation Manager Sub-Unit 33    -   Pn at least one navigation path—not represent graphically    -   SP preferred navigation path    -   WP wandering path—not represented graphically    -   PI point of interest    -   PI-OLD old point of interest    -   PPI potential point of interest—not represented graphically        Guiding Modes for Transmitting the Corresponding Navigation        Guiding Instructions-   T walkable tunnel-   93 milestones 93-   r radius with a predetermined length of the circle having the origin    the position of the Sensory unit 2-   94 intersection of the circle having the radius d1 with the    preferred navigation path SP-   S spatialized sound source-   d1 predetermined first distance of the spatialized sound source S in    respect to the Sensory unit 2-   d2 predetermined second distance

Example No. 1

FIGS. 14 to 17

-   84 initial point of interest=entrance door, as a particular example    of objects On from the plurality of objects On selected as point of    interest PI-   two entrance doors 84-01 and respectively 84-02, not represented    graphically as examples of objects On from the plurality of objects    On out of which it is selected the point of interest 84-   841 further point of interest=doorbell 841, as another particular    example of the point of interest PI-   83 dog, as a particular example of living being Ln from the    plurality of living beings Ln-   831 traffic lights-   832 pedestrian crossing-   942 walkable area, as a particular example of the walkable area WA-   941 non-walkable area, as a particular example of the non-walkable    area NA-   943 conditional walkable area, as a particular example of the    conditional walkable area CWA-   911 initial navigation path, as a particular example of the    preferred navigation path SP-   912 secondary navigation path, as another particular example of the    preferred navigation path SP-   922 walkable tunnel, as a particular example of the walkable tunnel    T

Group of Examples No. 2—FIGS. 18 to 28

-   85—windows, as particular examples of objects On from the plurality    of objects On that are-   potential points of interest PPI and as example of category of    objects On-   85-1 first window-   85-2 second window-   85-3 third window—not represented graphically-   85-4 fourth window—not represented graphically    Chosen extremities of the any of the windows 85: 85-E1 and 85-E2.-   S86 corresponding spatialized sounds of windows-   S86-1 corresponding to the first window 85-1—not represented    graphically-   S86-2 corresponding to the second window 85-2—not represented    graphically-   S86-3 corresponding to the third window 85-3—not represented    graphically-   S86-4 corresponding to the fourth window 85-4—not represented    graphically    Particular Examples of Spatialized Sounds-   S86 f spatialized sound having a particular frequency—not    represented graphically-   S86 f-1 spatialized sound having a particular frequency    corresponding to the window 85-1-   S86 f-2 spatialized sound having a particular frequency    corresponding to the window 85-2-   S86 f 1, and S86 f 2 spatialized sound sources being encoded with    different frequency that virtually moves on the contour of the    window 85-   S86 p spatialized sound having a particular pulse—not represented    graphically-   S86 t spatialized sound having a particular time characteristic—not    represented graphically-   S86 t-1, and S86 t-2 spatialized sound having different time    characteristics for representing the shape of the frames of the two    windows 85-1 and 85-2.-   S86 t 11-1, S86 t 12-1, spatialized sounds having different time    characteristics virtually moving on the contour of the window 85-1    for representing the shape of the exterior frame of the window 85-1-   S86 t 21-1, S86 t 22-1, spatialized sounds having different time    characteristics virtually moving on the contour of the window 85-1    for representing the shape of the interior frame of the window 85-1-   S86P punctiform sound,-   S86P1, and S86P2 spatialized punctiform sounds virtually moving on    the contour of the window-   S86P-E1, and S86P-E2 two spatialized punctiform sounds corresponding    to the extremities of the window 85-E1 and 85-E2.-   S86L linear sound-   S861, and S862 spatialized sounds virtually moving in an angled    pattern within the space between the contour of the interior frame,    and the exterior contour of the window 85.-   t₀ starting point, and t_(final) end point in time of spatialized    sound sources virtually moving on the contour of the window 85    detailed description and examples of realization

With reference to FIG. 1, the wearable device 1 comprises the following:a Sensory unit 2, a Processing and control unit 3, a Feedback unit 4, aUser commands interface 5.

The wearable device 1 comprises two hardware units not representedgraphically: a Power storage unit 6, and a memory M.

Throughout the invention, it shall be understood that the visuallyimpaired person is wearing the wearable device 1 and that the wearabledevice 1 is switched on. Therefore, any reference in the description,claims and drawings to the wearable device 1 or to the Sensory unit 2shall be understood as including a reference to the position of thevisually impaired person. For simplicity, throughout the invention, thevisually impaired user shall be referred to as “he”, encompassing allgender situations.

Details about the configuration and location of the hardware units willbe given in the section of the description that relates to theconfigurations of the wearable device 1.

For a better understanding of the method, the basic components of thehardware units are briefly described keeping the pace together with thedisclosure of the method.

The Sensory unit 2 is placed on the head of the visually impaired userand comprises basic sensors:

-   -   a Camera 21,    -   a Depth sensor 22,    -   an Inertial Measurement unit 23    -   a Sound localisation sensor 24

The Processing and control unit 3 comprises:

-   -   a Sensory fusion sub-unit 30,    -   a Live Map sub-unit 31,    -   a Relationship Manager sub-unit 32,    -   a Navigation Manager sub-unit 33,    -   a User commands interface Manager sub-unit 34,    -   a Feedback Manager sub-unit 35,    -   a Sound representation sub-unit 36,

The Sensory fusion sub-unit 30 comprises:

-   -   a Localisation module 301,    -   a Walkable Area Detection module 302,    -   an Orientation Computation module 303,    -   a Sound Direction Localisation module 304,    -   a Sound Classification module 305,    -   an Object 2D Characteristics Extraction module 306,    -   an Object 3D Characteristics Fusion module 307,    -   an Object Sound Characteristics Fusion module 308,

FIG. 2 illustrates a preferred location of the Sensory unit 2 on theforehead of the visually impaired user. Throughout the detaileddescription and the figures, it is exemplified the preferred location onthe forehead of the user. The person skilled in the art shall understandthat the invention is not limited to placing the Sensory unit 2 on theforehead.

The method according to the invention includes 4 steps. The four stepswill be firstly described briefly in their succession. Then, steps 2, 3and 4 will be detailed.

S1 The sensory unit 2 of the wearable device 1, placed on the head ofvisually impaired user, acquires data from the environment of thevisually impaired user.

For this purpose, the sensory unit 2 senses from a field of view 20having as origin the position of the sensory unit 2.

S2 The data sensed by the sensory unit 2 is sent to the Sensory fusionsub-unit 30.

The Sensory fusion sub-unit 30 fuses the data acquired from the sensoryunit 2 by data processing algorithms that include filtering, smoothing,and artificial intelligence-based algorithms, and then sends the fuseddata to the Live map sub-unit 31 of the Processing and control unit 3.

Further on, the Live Map sub-unit 31 creates, repeatedly updates andstores a Live Map 310. The Live Map 310 comprises three categories ofdata:

1. Live Map determinations that are generated based on fused datareceived from the Sensory Fusion sub-unit 30,

2. Live Map determinations that are generated based on a plurality ofrelationships Rn between the plurality of objects On and/or theplurality of living beings Ln,

3. Live Map determinations based on a free area A.

The Live Map 310 is a database stored in the memory M. Throughout theinvention, the update and store of the data in the Live Map 310 shallinclude the update and store of the Live Map 310 in the memory M. Theway the Live Map 310 is stored is outside the scope of the invention.

FIG. 3A and FIG. 3B schematically shows the field of view 20 having asorigin the position of the sensory unit 2, while FIG. 4 schematicallyshows the content of the field of view 20 and of the Live Map 310 ascompared with the real environment.

In an embodiment of the present invention, the Live map 310 alreadyexists in the memory M. In this case, the territorial range of the LiveMap 310 is determined by the content stored in the past in the Live Map310. As it can be seen from FIG. 4, the Live Map 310 includes objectsthat are no longer in the field of view 20, but they were in the fieldof view 20 in the past. The Live Map 310 of FIG. 4 includes an old pointof interest PI-OLD, that is a point of interest in the past.

1. The Live Map determinations based on the fused data received from theSensory Fusion sub-unit 30 include the following determinations:

-   -   A position and an orientation of the sensory unit 2,    -   A plurality of objects On. For each object On the determinations        refer to:        -   its position,        -   its physical, acoustical, and chemical characteristics,        -   its orientation,        -   the prediction of its future position in a predetermined            unit of time,    -   A plurality of living beings Ln. For each living being Ln the        determinations refer to        -   its position,        -   its biological and acoustical characteristics,        -   its orientation, current activity and mood status,        -   the prediction of its future position in the predetermined            unit of time.            2. The Live Map determinations based on the plurality of            relationships Rn between the plurality of objects On and/or            the plurality of living beings Ln are received from a            Relationship Manager sub-unit 32 of the Processing and            control unit 3.            The Relationship Manager sub-unit 32 imports the most recent            updates from the Live Map 310 by querying the Live Map            sub-unit 31 for updates in the Live Map 310.            Computations are based on predetermined relations            requirements, comprising at least:    -   predetermined parent-child relations, and    -   predetermined conditional relations.        For simplicity, throughout the invention:    -   the term “relations” is used as equivalent wording for the        plurality of relationships Rn, and    -   the plurality of relationships Rn, alternatively called        relations, refer to both static and dynamical relationships.

After carrying out the computations, the Relationship Manager sub-unit32 sends the updated relations as result of the computations to the Livemap sub-unit 31 to store same in the Live Map 310.

Details regarding the generation of the plurality of relationships Rnare given in the section related to S.2.2 below.

3. The Live Map determinations based on the free area A

The free area A is defined as an ensemble of areas on a ground notoccupied by the plurality of objects On and the plurality of livingbeings Ln.

Said free area A is divided into three categories:

-   -   A walkable area WA that satisfies a set of permanent        predetermined walkable area requirements, defining the walkable        area WA as an area on which the visually impaired user can walk        on without being injured,    -   A conditional walkable area CWA that satisfies said set of        permanent predetermined walkable area requirements, and        satisfies at least one predictable conditional walkable area        requirement, and    -   A non-walkable area NA that does not satisfy neither the set of        permanent predetermined walkable area requirements, nor the at        least one additional predictable conditional walkable area        requirement.

In S3, automatically or in response to a request from the visuallyimpaired user, a Navigation Manager sub-unit 33 of the Processing andcontrol unit 3 determines, repeatedly updates and stores in the memoryM, one or more navigation paths Pn for the visually impaired user tonavigate from the current position of the sensory unit 2 to a point ofinterest PI selected among the plurality of objects On and/or theplurality of living beings Ln.

The term “navigation” shall be understood in this invention asencompassing:

-   -   the walking of the visually impaired user towards an object On        or a living being Ln,    -   the everyday gestures and actions made with one or both hands or        limbs of the visually impaired user for finding and reaching        various objects such as the toothbrush, the doorknob, etc.        Non-Limiting Examples of Navigation Paths Pn Include:    -   Navigation paths Pn for walking outdoors,    -   Navigation paths Pn for walking indoors,    -   Navigation paths Pn for reaching a large variety of objects for        a variety of purposes: from small objects such as a comb to        large objects such a plane.

The Navigation Manager sub-unit 33 repeatedly selects, automatically orin response to the request from the visually impaired user, onepreferred navigation path SP. If only one navigation path Pn wasdetermined, then the preferred navigation path SP is the navigation pathPn. If two or more navigation paths Pn were determined, the NavigationManager sub-unit 33 repeatedly selects one of them as the preferrednavigation path SP.

The preferred navigation path SP is repeatedly sent by the NavigationManager sub-unit 33 together with associated navigation guidinginstructions, to a Feedback Manager sub-unit 35 of the Processing andcontrol unit 3.

In order to determine one or more navigation paths Pn, the NavigationManager sub-unit 33 queries the Live Map 310 in order to check if atleast two navigation path requirements are met. The first navigationpath requirement is that all navigation paths Pn—thus including thepreferred navigation path SP, must pass through the walkable area WAand/or on the conditional walkable area CWA.

The second navigation path requirement is to meet a set of safetyrequirements in respect to the plurality of objects On and/or theplurality of living beings Ln positioned or predicted to be positionedin the proximity of the at least one navigation path Pn in thepredetermined unit of time. The proximity is predetermined, for exampleat 0.3 m from the position of the wearable device 1. The set of safetyrequirements includes at least one non-collision requirement and atleast one non-aggressivity requirement. Other safety requirements may bedefined for various specific needs arising either from the needs of thevisually impaired person e.g., elderly person, or from thecharacteristics of the environment where the visually impaired personusually lives, e.g., a densely populated urban area or from both.

The non-collision requirement means that the individual paths ofplurality of objects On and/or the plurality of living beings Ln mustnot collide with the at least one navigation path Pn.

The non-aggressivity requirement means that the mood of the plurality ofliving beings Ln must not anticipate an aggressive action directedagainst the visually impaired user.

Other navigation path requirements may be defined by the user such asbut not limited to the requirement to avoid crowded areas or to avoidpassing through zones with slopes higher than a predetermined value.

The navigation path requirements are predetermined and stored in thememory M. They are applied by the Navigation Manager sub-unit 33. Thevisually impaired user can set other predetermined navigation pathrequirements by means of the User commands interface Manager sub-unit34. When the selected navigation path SP passes through the conditionalwalkable area CWA, the Navigation Manager sub-unit 33 sends to theFeedback Manager sub-unit 35 an associated navigation guidinginstruction associated to said at least one predictable conditionalwalkable area requirement.

The determination of the at least one navigation path Pn is initiatedeither automatically by the Navigation Manager sub-unit 33 or byreceiving from the visually impaired user of an initiation request,

In case the Navigation Manager sub-unit 33 determines two or morenavigation paths Pn, the selection of the preferred navigation path SPis carried out either automatically by the Navigation Manager sub-unit33 or by receiving by said Navigation Manager sub-unit 33 of a selectionrequest from the visually impaired user.

The Navigation Manager sub-unit 33 can be configured such that, bydefault, the selection of the preferred navigation path SP be carriedout either automatically by the Navigation Manager sub-unit 33, oraccording to the selection request from the visually impaired user.

When carried out automatically by the Navigation Manager sub-unit 33,the selection of the preferred navigation path SP is based on applying aset of pre-determined path selection criteria including cost criteria,cost-time to destination criteria, comfort criteria. The application ofthe path selection criteria is carried out according to prior art.

The requests made by the visually impaired user are made by using hapticmeans 51 or audio means 52 of a User commands interface 5. Theserequests are received by the Navigation Manager sub-unit 33 via a Usercommands interface Manager sub-unit 34 of the Processing and controlunit 3.

In S4 the Feedback Manager sub-unit 35 guides the visually impaired useralong the preferred navigation path SP, by using guiding modes fortransmitting each associated navigation guiding instruction as receivedfrom the Navigation Manager sub-unit 33.

The guiding modes are sent by the Feedback Manager sub-unit 35 to aFeedback unit 4 of the Processing and control unit 3. Each navigationguiding instruction comprises haptic and/or auditory cues.

The Guiding Modes are:

-   -   either by haptic cues by using haptic feedback actuators 41 of        the Feedback unit 4, configured for placement on the forehead of        the visually impaired user, or    -   by auditory cues by using auditory actuators 42 of the Feedback        unit 4, configured for placement adjacent to one or both ears of        the visually impaired user, or    -   by combining the haptic cues with the auditory cues.

The selection of the guiding modes for each associated navigationguiding instruction is carried out by the visually impaired user by theUser commands interface 5 and through user commands that are received bythe Feedback Manager sub-unit 35 via the User commands interface Managersub-unit 34.

Details regarding S2—with reference to FIG. 5

The Live Map 310 can be compared with a multi-layer cake, as severallayers of information are added from the first sub-step until the lastsub-step as described below. With each layer, the Live Map 310 acquiresa higher level of detail and accuracy. The creation is continuous,having as result the continuous update and continuous storage of theLive Map 310.

The advantage of creating multiple information layers in the Live Map310 is related to the ease of use of understanding and accessing thedata. As each individual layer contains specific information which isrelevant to certain other components of the system, this facilitatesfaster access to the information.

S2.1. The Live Map sub-unit 31 creates and updates the Live Map 310 byrepeatedly determining the position and orientation of the sensory unit2, the position and orientation, and characteristics of the plurality ofobjects On, of the plurality of living beings Ln, based on the fuseddata received from the Sensory Fusion sub-unit 30, and repeatedly sendsthe created and updated Live Map 310 to a Localisation module 301 of theSensory Fusion sub-unit 30,

S2.2. The Relationship Manager sub-unit 32 repeatedly generates andupdates a plurality of relationships Rn between the plurality of objectsOn and/or the plurality of living beings Ln based on the data acquiredfrom the Live Map 310 comprising applying a set of the predeterminedrelations requirements. The plurality of relationships Rn, repeatedlyupdated, are repeatedly sent to the Live Map 310, thus updating the LiveMap 310 content as outputted from S2.1. with the layer referring to thecontent of the plurality of relationships Rn,

S2.3. The Localisation module 301 repeatedly localizes the position andorientation of the sensory unit 2 with respect to the plurality of theobjects On, and, respectively to the plurality of living beings Ln ofthe Live Map 310 using localisation algorithms applied to the datareceived from the sensory unit 2 and data from the of the Live Map 310.The localisation of the position and orientation of the sensory unit 2is repeatedly sent to a Walkable Area Detection module 302 of theSensory fusion sub-unit 30, thus updating the Live Map 310 content asoutputted from S2.2 with the layer referring to the localisation data ofthe position and orientation of the sensory unit 2 in respect to theplurality of the objects On, and, respectively to the plurality ofliving beings Ln.

S2.4. The Walkable Area Detection module 302 repeatedly determines thefree area A, based on:

i) the data received from the sensory unit 2,

ii) the data received from the Localisation module 301,

iii) the set of permanent predetermined walkable area requirements, and

iv) the at least one predictable conditional walkable area requirementscalculated and stored in the memory M of the Walkable Area Detectionmodule 302.

The components of the free area A repeatedly updated, are repeatedlysent to the Live Map 310, thus updating the Live Map 310 content asoutputted from S2.3 with the layer referring to the components of thefree area A.

S.2.5. The updated Live Map 310 is repeatedly stored in the memory M.

S.2.1. Details regarding the Live map determinations based on fused data

The Orientation Computation module 303 determines the current positionand orientation of the Sensory unit 2 of the wearable device 1, of theplurality of the objects On and the plurality of living beings Ln inrespect to the sensory unit 2 based on the inertial movement dataprovided by the Inertial Measurement unit 23. For this purpose, theOrientation Computation module 303 applies an orientation computationalgorithm that calculates the orientation of the system on the 3 axes(pitch, roll, yaw) and for the 3D positioning of objects since theCamera 21 and Depth sensor 22 reveal where are the detected objects Onin respect to the Camera 21, but not how they are oriented in respect tothe ground.

The Object 2D Characteristics Extraction module 306 provides thepixel-wise segmentation of the 2D images acquired from the Camera 21,and detects in the pixel-wise segmented 2D images each object On of theplurality of the objects On, and each living being Ln of the pluralityof living beings Ln placed in the field of view 20, and determines theirrespective position in 2D coordinates, and their respective physicalcharacteristics.

The Object 2D Characteristics Extraction module 306 uses an Object 2DCharacteristics Extraction Algorithm that combines several actions:

-   -   Object Detection to determine the type of objects On or living        beings Ln, their 2D position and their relative 2D size and        their 2D centre, their 2D motion vector in respect to the Camera        21 by comparing the data between several subsequent images;    -   Object Pose Detection to determine the orientation of the object        in 2D coordinates,    -   Skeleton Pose detection to determine the posture of living        beings Ln by skeleton orientation, which is used to understand        activities of living beings Ln, such as run, sit, cough, etc.        and status, for example sleeps, is awake, etc.    -   Face feature detection to determine empathic status, such as        smiles, laughs, cries, etc., and also activity status: sleeping,        awake, tired, etc.    -   Object Characteristics Determination which comprises algorithms        for various aspects, such as:    -   the degree of occupancy of a chair, a handlebar, a fridge or a        room—for example by comparing how much of it is visible versus        how it should be according to a customary image;    -   the degree of 2D filling of a container for example in the case        of transparent containers;    -   the degree of dirtiness of a product for example by comparing of        the objects On captured by the Camera 21 with a known images of        clean similar objects On, and computing differences. Further on,        the Object 3D Characteristics Fusion module 307 receives data        from the Object 2D Characteristics Extraction module 306, from        the Orientation Computation module 303, and from the Depth        sensor 22, and determines further detailed information about 2D        information received from the Object 2D Characteristics        Extraction module 306 regarding the plurality of the objects On,        and plurality of the living beings Ln.

Thus, the Object 3D Characteristics Fusion module 307 determines theposition each of the objects On in respect to the Sensory unit 2 in 3Dcoordinates, their physical characteristics, such as dimensions,composition, structure, colour, shape, humidity, temperature, degree ofoccupancy, degree of cleanliness, degree of usage, degree of wear,degree of stability, degree of fullness, degree of danger, and theirorientation in respect to the Sensory unit 2, and the future position atpredetermined moments in time in 3D coordinates based on the vector ofmovements, respectively. The Object 3D Characteristics Fusion module 307also determines data regarding position of each of the living beings Lnin 3D coordinates, their physical characteristics, like height, theirskeleton pose orientation, and the prediction of its future position inthe predetermined unit of time, respectively. Based on skeleton poseorientation, facial expression and their physical characteristics theObject 3D Characteristics Fusion module 307 determines the currentactivity and mood status of each of the living beings Ln.

The Sound Direction Localisation module 304 determines the direction ofthe plurality of sound streams expressed in 3D coordinates emittedrespectively by each of the plurality of the objects On and theplurality of living beings Ln based on the data received from the Soundlocalisation sensor 24.

In one embodiment of the method, the direction of the plurality of soundstreams is determined by comparing the differences of a sound streambetween microphones of the Sound localisation sensor 24 while knowingthe position of the microphones. The Sound Direction Localisation module304 triangulates the source of the sound stream coming, detecting thedirection from which the sound stream comes.

Each of the plurality of sound streams whose direction has beendetermined by the Sound Direction Localisation module 304 is classifiedinto sound types by means of the Sound Classification module 305.

The Object Sound Characteristics Fusion module 308 adds acousticalcharacteristics to each of the plurality of the objects On and theliving beings Ln for which the 3D coordinates have been determined basedon the classified sound types determined by the Sound Classificationmodule 305.

Then, the Object Sound Characteristics Fusion module 308 sends all fuseddata to the Live Map sub-unit 31 in order to be stored in the Live Map310.

S.2.2. Details Regarding the Generation of the Plurality ofRelationships Rn

The Live Map determinations based on the data received from theRelationship Manager sub-unit 32 provide further detailed informationdefining the environment of the visually impaired user. In this way theRelationship Manager sub-unit 32 of provides more accurate and detailedinformation about the objects On and the living beings Ln fulfilling theinvention's objective of a safer navigation of the visually impaireduser and a more concrete navigation goal, the latter being defined inthe invention as the point of interest PI.

The algorithms used by the Processing and control unit 3 include but arenot limited to: Object Detection, Object Pose Detection, ObjectCharacteristics Determination. The algorithms used by the processing andcontrol unit 3 define as item:

-   -   each object On from the plurality of objects On,    -   each living being Ln from the plurality of living beings Ln,    -   each component part of each object On, for example: the leg of        the chair,    -   each part of each living being Ln.

For example, in case of the object On is a four-leg chair, the chair isdefined as a separate item from each one of its four legs.

The degree of itemization is predetermined being outside the scope ofthe invention.

The processing and control unit 3 creates clusters of objects based ontheir physical relationships. Thus, predetermined parent-child relationsconnect the separate items so that they can form objects On, livingbeings Ln or ensembles between more than two objects On, more than twoliving beings Ln or objects On and living beings Ln. For example: thedoor handle belongs to the door. Both the door handle and the door areitems. The main difference between the items on one hand, and theobjects On and living beings Ln on the other hand is that the objects Onand living beings Ln correspond to the usual expectation of the peopleabout what an object and a living is, whereas for the algorithms all theobjects On and the living beings Ln as well as their components aretreated as items.

The predetermined conditional relations refer to connecting the separateitems only if a condition is satisfied, for example the pedestriancrossing is a conditional walkable area, conditioned on the colour ofthe traffic light.

The Relationship Manager sub-unit 32 uses the data from the Live Map 310to compute possible relations using specific algorithms.

For parent-child relations, non-limiting examples of algorithms are asfollows:

-   -   Physical proximity: If the items corresponding to the objects On        or to the living beings Ln are in physical proximity, they form        a parent-child relation:    -   If a cap and an open water bottle are close by, the water bottle        becomes the parent of the cap which is the child,    -   If a keyboard or mouse is close by to a computer, they become        the child towards the computer which becomes the parent,    -   If a living being pet such as a dog or cat is detected in close        proximity to a living being human, the pet becomes the child of        the human which becomes the parent,    -   If a door handle is close to a door, it becomes the child of the        door. Likewise, the doors of the vehicles are children for the        vehicle.    -   Physical proximity with containment: If the items corresponding        to the objects On or to the living beings Ln are in physical        proximity, and one object or living being is contained in the        other object they form another parent-child relation:    -   If fish are detected in close proximity and contained in a fish        tank, the fish become the children of the fish tank which        becomes the parent,    -   If liquid is detected in a transparent container, the container        becomes the parent and the liquid the child,    -   If seats are detected in the proximity of a bus and the seats        are contained inside the bus, the seats become the children of        the bus,    -   Physical proximity with intersection: If objects On are in        physical proximity and one or multiple of their planes are        intersecting, they form another parent-child relation:    -   If doors and walls are detected in close proximity and their        planes are matching, the door becomes a child to the wall,        Creation of New Items Based on Detected Relations:    -   Physical proximity with intersection: if items corresponding to        objects On are in close proximity, they intersect, they create a        new item type and be allocated as the child of that item:    -   If a floor, a roof and multiple walls are detected in proximity        and intersecting—all being items, they form a room, which is        another item, and become children to the room,    -   If multiple rooms are generated, in proximity and intersecting        they create a floor, and become children to the floor,    -   If one or multiple floors are generated, they create a building        and become children to the building. For conditional        relationships non-limiting examples of algorithms are as        follows:    -   Physical proximity: if items corresponding to objects On are in        close proximity they form a conditional relationship:    -   If the doorbell is detected in proximity of the door, they form        a conditional relationship: one must ring the doorbell before        entering the door,    -   If a keyhole is detected in proximity of the door handle, they        form a conditional relationship: one must unlock the door before        operating the door handle,    -   Physical proximity with orientation: if items corresponding to        objects On are in close proximity, they are oriented in a        certain way, they form another conditional relationship:    -   If a conditional walkable area CA such as a pedestrian crossing        is detected, and a pedestrian traffic light oriented towards the        pedestrian crossing is detected, the conditional walkable area        CA is conditioned by the colour of the detected traffic light.

Depending on the type of object On or living being Ln, certainproperties are transmissible from a parent to a child, for example:

-   -   If the door handles are the children of doors which are the        children of a car, when the car moves, even if the door handles        or doors are no longer in the field of view 20 of the wearable        device 1, their position will be updated, even if the car        position has changed meanwhile.

All parameters used in the algorithms for establishing the plurality ofrelationships Rn are pre-determined: for example, for determiningphysical proximity predetermined ranges of distances are used.

S.2.3. Details Regarding the Localisation of the Position andOrientation of the Sensory Unit 2

The Localisation module 301 repeatedly determines current position andorientation of the sensory unit 2 of the wearable device 1 and of theplurality of the objects On and living beings Ln in respect to thesensory unit 2, in the 3D coordinates, on the current Live Map 310 bymeans of localisation algorithms applied to the data acquired from aCamera 21, the Depth sensor 22, an Inertial Measurement unit 23 of theSensory unit 2.

The results of the localisation are sent to the Walkable Area Detectionmodule 302 which determines the components of free area A.

S.2.4. Details Regarding the Determination of the Components of the FreeArea A.

The set of permanent predetermined walkable area requirements comprisescategories that are predetermined for each visually impaired user,taking into consideration various general safety and comfortrequirements.

The Set Comprises at Least the Following Two Categories:

-   -   geometric predetermined walkable area requirements refer to the        geometry of the space virtually occupied by the visually        impaired user. Thus, a virtual cuboid is imagined having its        three dimensions adjusted to the dimensions of the visually        impaired user. The virtual cuboid ensures protection of the        visually impaired user against injury when said visually        impaired user is standing still or is moving. Non-limiting        examples of requirements from this category are as follows: the        oscillations of the level of the ground must not exceed a        predetermined height, a beam placed at a certain distance from        the ground is considered dangerous if the distance is under a        predetermined threshold, etc.    -   surface predetermined walkable area requirements: certain ground        surface types are excluded such as but not limited to: water        film that exceeds a predetermined width, such as 5 cm; ice; mud;        streets and roads.

The conditional walkable area CWA does satisfy the set of permanentpredetermined walkable area requirements and must satisfy in additionthe at least one predictable conditional walkable area requirement.

The set of permanent walkable area requirements as well as the at leastone predictable conditional walkable area requirement are predeterminedfor each visually impaired user and stored in the memory M. The WalkableArea Determination module 302 applies said requirements to the data itreceives from the Camera 21 and Depth sensor 22 on one hand and from theLocalisation module 301 on the other hand, said data received from theLocalisation module 301 including the updates of the relations asreceived from the Relationship Manager sub-unit 32 and stored in theLive Map 310.

In another preferred embodiment, parts of the Live Map 310 aredownloadable form the internet from any geographical maps site, saidparts referring to the layers described in S2.1 to S.2.4 and taking intoaccount that, depending on the geographical maps site from where map isdownloaded, the information of each layer can be partial or complete.The download from the internet is carried out using a Communication unit7, not represented graphically, connected to the internet. In this case,the Localisation module 301 localizes the position and orientation ofthe sensory unit 2 on the downloaded Live map 310.

In case there is a previously stored Live Map 310 in the memory of theLive map sub-unit 31, the Live Map 310 is created based on the Live Mapdeterminations of the previously stored Live Map 310.

In case a previously stored Live Map 310 exists in the memory M of thewearable device 1, either because it was determined by the Live Mapsub-unit 31 previously or because it was downloaded from the internet orboth of them, the determinations based on the data received from thesensory unit 2 start with step 2.3 by the identification withinpreviously stored Live Map 310 of the current position and orientationof the sensory unit 2 by means of localisation module 301, andidentification of the free area A, including the walkable area WA andthe conditional walkable area CWA by the Walkable Area Detection module302 by means of localisation algorithms applied to the data receivedform the sensory unit 2.

Further on, the Live map 310 is repeatedly updated with additionalinformation described in S2.1 to S.2.2 and the remainder of step 2.3 andsteps 2.4 and steps 2.5. are carried out as described above.

In a preferred embodiment, the Live Map (310) is updated by the Sensoryfusion sub-unit (30) using Simultaneous Localisation and Mapping SLAMalgorithms.

The SLAM algorithms are in particular advantageous since they use aniterative process to improve the estimated position with the newpositional information. The higher the iteration process, the higher thepositional accuracy. This cost more time for computation andhigh-configuration hardware with parallel processing capabilities of theprocessing units.

In another preferred embodiment the SLAM algorithms used are visual SLAMalgorithms which have the benefits of providing vast information, beingcheap and easy to implement since may be used passive sensors andcomponents having extremely low size, weight, and power SWaP footprint.

The invention, as disclosed so far, refers to the cases where the pointof interest PI is known to the visually impaired user before sending theinitiation request.

In other cases, the visually impaired user has not sufficientinformation about the point of interest PI before sending the initiationrequest. Typical examples are when he arrives in a new environment, orwhen something has changed in the known environment, such as the usualplaces of the seats.

One example is when the visually impaired user enters a new room thathas four windows. He wants to open a window. But which one of the fourwindows to select as point of interest PI? Or the visually impaired userenters a conference room where there are, say 30 occupied seats and 10free seats. Which of the 10 free seats to choose as point of interestPI?

To encompass these cases where the visually impaired user needsadditional information from his environment in order to select the pointof interest PI before sending the initiation request, in anotherpreferred embodiment, when the point of interest PI is not known by thevisually impaired user, a sub-step 3-0 is carried out before all theother sub-steps of step 3:

In S.3-0.1. the visually impaired user sends an information request to aSound representation sub-unit 36 of the Processing and control unit 3regarding at least one object On selected from the plurality of objectsOn or at least one living being Ln selected from the plurality of livingbeings Ln, said at least one object On or at least one living being Lnas a potential point of interest PPI for the visually impaired user. Anexample of at least one object On selected from the plurality of objectsOn is a group of windows from a selected room, which may be named“window”.

The term “potential” means that any of the objects On from the group ofobjects On may be selected as initial point of interest PI.

The Sound representation sub-unit 36 is:

-   -   either a self-contained sub-unit connected to Live Map sub-unit        31, to the Feedback Manager sub-unit 35, and to the User        commands interface sub-unit 34, or    -   a sub-unit of the Navigation Manager sub-unit 33, as it is        represented for simplicity in FIG. 1, or    -   a module of the Feedback Manager sub-unit 35.

Taking the example of the room with four windows, the visually impaireduser sends an information request named “window” through the Usercommands interface 5 to the Sound representation sub-unit 36 that he isinterested to learn how many windows are in the room, their position inthe room, the size or the shape of the windows, the position of theirhandles. The window is in this example the potential point of interestPPI. The information request refers to a predetermined area of interestwhich is in the proximity of the place where the visually impaired userstands at the moment when he sends the information request, which inthis case is the room. The information request is transmitted by theUser commands interface 5 to the Sound representation sub-unit 36 viathe User commands interface Manager sub-unit 34, just like theinitiation request and the selection request.

In S.3-0.2. the Sound representation sub-unit 36 extracts from the LiveMap 310 the information regarding the selected at least one particularobject On or at least one particular living being Ln and represents saidat least one particular object On or at least one particular livingbeing Ln, respectively, as corresponding spatialized sounds andtransmits same to the Feedback Unit 4, via the Feedback Manager sub-unit35, when the Sound representation sub-unit 36 is not part of saidFeedback Manager sub-unit 35.

The representation in spatialized sounds is generated by means of theSound representation sub-unit 36 by encoding the classified sound typesof the selected objects On or, respectively, selected living beings Lnbased on predetermined spatialized sounds criteria.

The non-limiting and non-exhaustive examples of the predeterminedspatialized sounds criteria are:

-   -   binaural virtualization of the sounds depending on specific        features of the objects On from said specific category of        objects On or specific of living beings Ln from said specific        category of living beings Ln,    -   variation of the frequency, amplitude, period, frequency        components, fill factor of the spatialized sounds or duration,        and repetition spatialized sounds depending on the distance        relative to the visually impaired user of said objects On living        beings Ln.

The type of encodings of the classified sound types of the selectedobjects On or, respectively, the selected living beings Ln based onpredetermined spatialized sounds criteria is chosen based on testingprocedures determining the ability of the user to distinguish varioustechnical features of the sounds.

The visually impaired user is able to localize each spatialized soundusing natural capabilities of the human beings to process soundsemanating from sound sources and following adequate training with thewearable device 1.

The localization of the spatialized sounds is carried out in threespatial dimensions:

-   -   horizontal: the azimuth of the wearable device 1 essentially        corresponding to the azimuth of the forehead of the visually        impaired user,    -   vertical: the elevation, measured from the ground until the        wearable device 1 essentially corresponding to the elevation of        the forehead of the visually impaired user,    -   the distance range or the near-far dimension, measured from the        standing point of the sensory unit 2.

In S. 3-0.3, the visually impaired user selects the point of interest PIfrom said specific plurality of objects On or, respectively, from saidplurality of living beings Ln and transmits the corresponding selectionto the Navigation Manager sub-unit 33.

The group of examples No. 2 details the matter of the soundrepresentation.

In some situations, the point of interest PI is not in the Live Map 310,for example, when the visually impaired person arrives to a newdestination.

In this case, the Live Map unit 31 sends to the Navigation Managersub-unit 33 and to the User commands interface Manager sub-unit 34 theconfirmation that the point of interest PI is not in the Live Map 310.The method has an additional sub-step in S3 before determining,repeatedly updating and storing the at least one navigation path (Pn):

S3-1 The Navigation Manager sub-unit 33 determines a wandering pathWP—not represented graphically, while S1 and S2 of the method arerepeated until the point of interest PI is found and stored in the LiveMap 310, said wandering path WP satisfying the at least two navigationpath requirements.

It is possible to determine the wandering path WP while the NavigationManager sub-unit 33 represents as corresponding spatialized soundsspecific category of objects On or said specific category of livingbeings Ln. Once the decision as to the selection of the point ofinterest PI is taken, the remainder of step 3 and the step 4 of themethod are carried out as disclosed.

Details Regarding S4

All the guiding modes have the purpose to keep the visually impaireduser, when navigating, on the preferred navigation path SP. Eachpreferred navigation path SP has its own degree of complexity thatcorresponds to the variety of navigating situations arising from reallife. The inventors thought to quantify the degree of complexity of thepreferred navigation paths SP by using scores corresponding topredetermined preferred navigation path SP complexity criteria, whichinclude both objective criteria and subjective criteria, the latterbeing the own interpretation of the visual impaired user of theobjective criteria: e.g. what is perceived as a long distance for aspecific visually impaired user is not perceived as long for othervisually impaired user, the same with noise or temperature of theimmediate environment.

Below are presented some non-limiting and non-exhaustive examples of thepredetermined preferred navigation path complexity criteria:

-   -   Width of the walkable area WA and of the conditional walkable        area CWA: it is different to navigate if only 10 cm width        walkable area WA than on a 3 m width walkable area WA,    -   Distance left until the point of interest PI,    -   The number of turns and the degree of each turn e.g. 30°, 75°,    -   The slope and/or the number of stairs,    -   Noise of the environment, because it may limit the use of audio        cues.

The haptic cues vary in duration, periodicity, intensity or frequency ofthe vibration according to predetermined preferred navigation pathcomplexity criteria.

The audio cues vary in frequencies, duration, repetition, intensity, or3D spatial virtualization according to the predetermined preferrednavigation path complexity criteria.

The variation of the haptic cues and, respectively audio cues, has theadvantage of adapting the guidance of the visually impaired user to thedegree of complexity of each preferred navigation path as quantified bythe predetermined preferred navigation path SP complexity criteria. Theadvantages of the variation of the characteristics of the haptic cuesand of the auditory cues as well as the possibility to combine hapticand auditory cues are as follows:

-   -   Provides a better guidance of the visually impaired user along        the navigation paths in terms of accuracy and security,    -   Provides the possibility to customize the guidance depending on        the predetermined preferred navigation path complexity criteria;    -   Provides a more comfortable navigation offering the visually        impaired user the possibility to take more decisions constantly        adjusting the cues to his needs.        Haptic Cues

The haptic cues are received through the haptic feedback actuators 41.The visually impaired user receives training before use of the wearabledevice 1 in order to associate each type of haptic cue with the specificguiding instruction.

With reference to FIG. 6, a minimum number of three haptic feedbackactuators 41 is needed, all placed on the azimuth of the forehead—thatis on the same horizontal, in order to ensure the minimum threedirections of guiding: forward, turn left, turn right.

-   -   Left haptic feedback actuators 411 mounted on the left part of        the forehead,    -   Right haptic feedback actuators 412 mounted on the right part of        the forehead,    -   Centre haptic feedback actuators 413 mounted on the centre of        the forehead.

Said haptic feedback actuators 41 include vibrating actuators andclose-range remote haptics such as ultrasonic haptic feedback actuators.

Vibrating actuators comprise a plurality of resonant actuatorsconverting the electric signals received from the Feedback Manager 35into forced vibrations felt on the forehead of the visually impaireduser, said vibrations associated with a specific guiding instruction.

A non-limiting example of vibrating actuator used in the invention is alinear resonant actuator. Each of the left haptic feedback actuators411, right haptic feedback actuators 412 centre haptic feedbackactuators 413 can comprise one or more linear resonant actuators.

Using the linear resonant actuators is advantageous for the inventionbecause of their known good haptic performance, their improvedefficiency at resonance compared with other vibrating actuators, theircapacity of optimizing power consumption and their small size whichallows configuring them for example in the form of a matrix, if morethan three directions of guiding are envisaged.

There are Two Types of Haptic Cues:

-   -   Temporal haptic cues are the cues received at equal or unequal        intervals of time, by using any haptic feedback actuators 41,    -   Spatiotemporal haptic cues, alternatively called haptic pattern        cues, have a temporal component combined with a spatial        component, namely a pattern that represents the direction in        which the visually impaired user must reorient, e.g. from the        bottom to the top or from the top to the bottom, or to the right        or to the left and so on. The tactile sensation of direction is        obtained, for example, by using the linear resonant actuators        because their improved efficiency at resonance enhances the        variation of the duration, periodicity, intensity or frequency        of the vibration of the haptic cues. The plurality of linear        resonant actuators outputs vibrations in a predetermined rapid        succession, one linear resonant actuator vibrating after another        in the direction in which the visually impaired user must        reorient, so that to give the visually impaired user the tactile        sensation of having the forehead dragged by someone in the        direction in which he must reorient. Non-limiting examples of        applying predetermined preferred navigation path complexity        criteria are given below:    -   The haptic cues are more intense and/or more frequent and/or        have higher frequency of vibration directly proportional with:        -   the degree of the deviation from the preferred navigation            path SP,        -   the amount of movement required for the visually impaired            user to take, to differentiate a turn of 90° from a turn of            only 30°, or climbing 10 stairs from climbing only 2 stairs.    -   The haptic cues have smaller duration and/or less intensity        and/or less speed of the vibration if the estimated time of        navigation to the point of interest PI is above a predetermined        time threshold in order to avoid fatigue of the visually        impaired user from receiving so many haptic cues. The types of        haptic cues are predetermined for each case depending on the        needs of the visually impaired user. An example of        predetermination of haptic cues is presented below for a better        understanding of the teaching of the invention, and not for        limiting same:    -   A first haptic cue for starting the navigation from the current        position of the sensory unit 2,    -   A second haptic cue for signalling that the visually impaired        user has deviated to the left from the preferred navigation path        SP,    -   A third haptic cue for signalling that the visually impaired        user has deviated to the right from the preferred navigation        path SP,    -   A fourth haptic cue for going forward,    -   A fifth haptic cue for turning left or right,    -   A sixth haptic cue for going up or down,    -   A seventh haptic cue for temporary stop when the navigation has        not ended, if the Navigation Manager sub-unit 33 detects that at        least one navigation path requirements is not met or if it        detects a conditional walkable area CA which requires sending a        navigation instruction to stop until the at least one        predictable conditional walkable area requirement is met.    -   An eighth haptic cue for signalling the end of the navigation as        the point of interest PI is reached.

Further haptic cues can be defined to accommodate other navigationsituations or requirements of the visually impaired user.

To ensure a more accurate guidance and to avoid at the same timeunnecessary overloading of the visually impaired user with haptic cues,it is possible to combine the types of haptic cues. E.g.:

-   -   for simple navigation instructions such as start/stop—the first,        the seventh, the eighth cue from the example above only temporal        cues can be used, whereas    -   for the complex navigation instructions, the spatiotemporal cues        can be used, because the guidance to turn left/right or to go        up/down is more accurate when applying the vibration criteria to        the haptic pattern cues than when using only temporal haptic        cues.

The assignment of each type of haptic cue to one or more from thefeedback actuators 41 used is predetermined.

Auditory Cues

Auditory cues are sounds perceptible by humans received through theauditory feedback actuators 42 in the ears of the visually impaireduser.

The auditory feedback actuators 42 are speakers, headphones orbone-conduction speakers converting the electric signals received fromthe Feedback Manager sub-unit 35 into sounds. The associated navigationguiding instructions received through the auditory feedback actuators 42are based on the principle of assigning a specific sound to eachassociated navigation guiding instruction.

With reference to FIG. 6, a minimum number of two auditory feedbackactuators 42 are used:

-   -   Left auditory feedback actuators 421, mounted in or around the        left ear,    -   Right auditory feedback actuators 422 mounted in or around the        right ear.

Each of the left auditory feedback actuators 421 and right auditoryfeedback actuators 422 can comprise a plurality of speakers, headphonesor bone-conduction speakers placed on the same azimuth.

The types of auditory cues are predetermined for each case depending onthe needs of the visually impaired user. An example of predeterminationof auditory cues is presented below for a better understanding of theteaching of the invention, and not for limiting same:

-   -   A first auditory cue for starting the navigation from the        current position of the Sensory unit 2,    -   A second auditory cue for signalling that the visually impaired        user has deviated to the left from the preferred navigation path        SP,    -   A third auditory cue for signalling that the visually impaired        user has deviated to the right from the preferred navigation        path SP,    -   A fourth auditory cue for going forward,    -   A fifth auditory cue for turning left or right,    -   A sixth auditory cue for going up or down,    -   A seventh auditory cue for temporary stop when the navigation        has not ended,    -   An eighth auditory cue for signalling the end of the navigation        as the point of interest PI is reached,

Further types of auditory cues can be defined to accommodate navigationsituations or requirements of the visually impaired user.

The assignment of each type of auditory cue to one or more from theauditory feedback actuators 42 is predetermined.

Considering the origin of the sounds, there are two types of sounds:

-   -   Simple sounds originating in the auditory feedback actuators 42,        used for simple associated navigation guiding instructions such        as start and stop,    -   Spatialized sounds originating from one or more spatialized        sound sources S, used for all the associated navigation guiding        instructions except for start and stop.

In one preferred embodiment, depicted in FIG. 8 A and FIG. 8 B, theguiding mode comprises a three-dimensional walkable tunnel T defined asa virtual tunnel of predetermined cross-section, having as horizontallongitudinal axis the preferred navigation path SP. The guiding mode ofS4 comprises specific haptic cues sent to the visually impaired userwhen the visually impaired user is approaching the virtual walls of thewalkable tunnel T.

The three-dimensional walkable tunnel T is determined by the NavigationManager sub-unit 33 at the same time with the preferred navigation pathSP, and then sent to the Feedback Manager sub-unit 35 together with thehaptic cues.

The advantage of the walkable tunnel T is that it allows a morecomfortable navigation of the visually impaired user with a largerdegree of liberty to the left and to the right defined by the virtualwalls of the walkable tunnel T.

The guiding cues are transmitted when the visually impaired user isreaching the virtual walls of the walkable tunnel T so that he returnswithin the space defined virtual walls of the walkable tunnel T. In someembodiments, apart from the guiding cues signalling the virtual walls ofthe walkable tunnel T, other guiding cues are transmitted to confirmthat the visually impaired user is navigating safely within the virtualwalls of the walkable tunnel T.

The cross-section of the walkable tunnel T is predetermined depending onthe plurality of the possible cross-sections along the preferrednavigation path SP and on the visually impaired user's preferences.

The example No. 1 details the guiding modes using the walkable tunnel T.

In another preferred embodiment, with reference to FIG. 9, the preferrednavigation path SP is divided into predetermined segments delimited by aplurality of milestones 93. Each segment has at one end a currentmilestone 93 and at the other end a subsequent milestone 93. The words“next” and “subsequent” have the same meaning.

The guiding mode of S4 comprises haptic cues and/or auditory cuessignalling the position of a next milestone 93 providing associatednavigation guiding instructions to the visually impaired user from acurrent milestone 93 to the subsequent milestone 93. When the visuallyimpaired user has already passed the subsequent milestone 93, saidsubsequent milestone 93 becomes the current milestone 93 and so on.

The length of the predetermined segments varies depending on thecomplexity and length of the preferred navigation path SP.

The length of each segment between two consecutive milestones 93 isinversely proportional with the predetermined preferred navigation pathcomplexity criteria: the more complex the preferred navigation path SP,the shorter each segment. The milestones 93 are more frequent in theportions that contain change of direction in either horizontal orvertical plane than in the portions of going straight.

The length of each segment between two consecutive milestones 93 isdetermined by applying the predetermined preferred navigation path SPcomplexity criteria, which means that the length of the segments alongthe preferred navigation path SP is not necessarily equal, as seen inFIG. 9.

The length of each segment can be calculated using scores correspondingto said predetermined preferred navigation path complexity criteria orcan be adapted dynamically for using artificial intelligence-basedlearning methods. For example, if the visually impaired user has somepreferred navigation paths SP that are repetitive and he selects guidingmethod by using milestones as favourite, it is convenient to use saidlearning methods to adapt dynamically the length of the milestones.

The Cues Used in the Guiding Mode from the Current Milestone 93 to theSubsequent Milestone 93 are:

-   -   haptic cues,    -   auditory cues, or    -   haptic and auditory cues.

A non-limiting example of using the haptic cues is as follows:

-   -   the first, the seventh, the eighth cues are temporal.    -   the second and the third cues are spatiotemporal signalling if        the visually impaired user is away from the preferred navigation        path SP,    -   the fourth cue is spatiotemporal signalling the next milestone        93 when going forward,    -   the fifth and the sixth are spatiotemporal signalling the next        milestone 93 defined in this case as the place where the        visually impaired user must reorient his direction of movement        on the horizontal or, respectively, vertical plane,

The variation of the duration, periodicity, intensity or frequency ofthe vibration of the haptic pattern cues is directly proportional to thepredetermined preferred navigation path complexity criteria and at thesame time they vary inversely proportional to the distance left untilthe subsequent milestone 93.

A non-limiting example of using the auditory cues is as follows:

-   -   the first, the seventh, the eighth cues are simple sounds,    -   all the other cues are spatialized sounds. The auditory feedback        actuators 42 repeatedly output the location in space of said        subsequent milestone 93 using the spatialized sound heard from        the position of said subsequent milestone 93 until the visually        impaired user has reached said subsequent milestone 93. Once        each milestone 93 reached, its spatialized sound is no longer        heard, it becomes the current milestone 93 and the spatialized        sound corresponding to the subsequent milestone 93 starts to be        heard and so on.

The spatialized sounds vary directly proportional in frequencies,duration, repetition, intensity, and 3D spatial virtualization accordingto the predetermined preferred navigation path complexity criteria andat the same time they vary inversely proportional to the distance leftuntil the subsequent milestone 93.

Using only auditory cues is advantageous in the situation when there isonly one subsequent milestone 93 that coincides with the point ofinterest PI: for example, if the visually impaired user needs to go fromthe sofa to the kitchen, in this case the kitchen being the only onesubsequent milestone 93. The spatialized auditory cue corresponds inthis case to the kitchen. Using auditory cues has the advantage ofsimplicity and predictability, because it provides the visually impaireduser the possibility to associate the distance left to be navigateduntil the subsequent milestone 93 with the corresponding auditory cueheard from the position of said subsequent milestone 93, which improveshis degree of orientation and feeling of safety when navigating. Usingonly auditory cues is preferred when the point of interest PI is knownto the visually impaired user and the distance to be travelled until thepoint of interest PI is short, for example for the navigation pathsinside the house.

When the guiding mode from the current milestone 93 to the subsequentmilestone 93 is by haptic and auditory cues, one between said haptic andauditory cues may be defined as primary and the other one as secondary,the secondary outputting cues only in special predetermined situation,such as for example the seventh cue instructing to stop and resume.

In another preferred embodiment, with reference to FIG. 10, it ispossible to combine the haptic cues of the guiding mode using thewalking tunnel T with the auditory cues of the guiding mode from thecurrent milestone 93 to the subsequent milestone 93. The associatednavigation guiding instructions sent by haptic cues aim to keep thevisually impaired user on the preferred navigation path SP and withinthe limits of the walkable tunnel T, whereas the auditory cues enablethe visually impaired user to estimate the distance left to be navigateduntil the next milestone 93. Combining the two guiding modes has theadvantage of combining the advantage of each of the guiding modes: thecomfort and the safety of the walking tunnel with the simplicity andpredictability of the guiding mode from the current milestone 93 to thesubsequent milestone 93. In another preferred embodiment, the guidingmode of S4 consists in haptic cues or auditory cues or haptic andauditory cues signalling the direction on the preferred navigation pathSP.

A Non-Limiting Example of Using the Haptic Cues is as Follows:

-   -   the first, the seventh, the eighth cues are temporal.    -   the second and the third cues are spatiotemporal signalling if        the visually impaired user is away from the preferred navigation        path SP,    -   the fourth cue is spatiotemporal signalling the direction        forward,    -   the fifth and the sixth cues are spatiotemporal signalling the        direction in which the visually impaired user must reorient his        direction of movement on the horizontal or, respectively,        vertical plane.

The haptic pattern cues are predetermined such that they give theimpression to the visually impaired user to be dragged by his foreheadconstantly towards the direction in which he moves by a person standingin front of him.

A non-limiting example of using the auditory cues is as follows:

-   -   the first, the seventh, the eighth cues are simple sounds,    -   all the other cues are spatialized sounds.

The spatialized sounds vary directly proportional in frequencies,duration, repetition, intensity, or 3D spatial virtualization accordingto the predetermined preferred navigation path complexity criteria. Thevisually impaired person, when navigating, follows the direction of thespatialized sound source S.

The main difference between the guiding mode based on signalling thedirection on the preferred navigation path SP and based on the guidingmode from the current milestone 93 to the subsequent milestone 93 refersto the variation of the features of the haptic pattern cues, andrespectively spatialized sounds:

-   -   in both guiding modes the haptic pattern cues, and respectively        spatialized sounds vary directly proportional according to the        predetermined preferred navigation path complexity criteria,    -   in case of the guiding mode from the current milestone 93 to the        subsequent milestone 93 there is an additional variation related        to the distance to the subsequent milestone 93, that does not        exist in the guiding mode based on signalling the direction on        the preferred navigation path SP depending on the distance to        the subsequent milestone 93.

The use of haptic cues or auditory cues signalling the direction on thepreferred navigation path SP is advantageous to be used in situationswhen the degree of complexity of the preferred navigation path SP islower than in the case of using the guiding mode from the currentmilestone 93 to the subsequent milestone 93 or the guiding mode of thewalking tunnel T. One such example is when the same preferred navigationpaths SP are used frequently. The advantage of the use haptic cues orauditory cues signalling the direction on the preferred navigation pathSP is that they produce less fatigue to the visually impaired user.

One non-limiting example of using haptic cues signalling the directionon the preferred navigation path SP is given in FIG. 11A. In thisexample, firstly is determining the way of signalling the direction onthe preferred navigation path SP by intersecting the preferrednavigation path SP with a circle having the origin the position of theSensory unit 2 and a radius r with predetermined length gets anintersection 94. The imaginary line connecting the origin of the Sensoryunit 2 with the intersection 94 gives the direction to be followed bythe visually impaired user, said direction being transmitted to him bysaid haptic cues or audio cues. The predetermined length of the radius ris set depending on the degree of complexity of the preferred navigationpath SP. In another non-limiting example of using auditory cuessignalling the direction on the preferred navigation path SP is given inFIG. 11B. In this example, the direction to be followed is establishedin the same way as in the example of FIG. 11A.

The spatialized sound source S is placed at a predetermined firstdistance d1 of the spatialized sound source Sin respect to the Sensoryunit 2.

In order to obtain flexibility in the guiding modes and to adapt saidguiding modes to the degree of complexity of the preferred navigationpath SP, the predetermined first distance d1 of the spatialized soundsource Sin respect to the Sensory unit 2 can be either smaller than thepredetermined length of the radius r—as depicted in FIG. 11B, or can beequal to or greater than it. It is possible to combine haptic cues withauditory cues, the combination not being represented graphically.

In another preferred embodiment, with reference to FIG. 12, the auditorycues are spatialized sounds originating from a spatialized sound sourceS that virtually travels along a predetermined second distance d2 on thepreferred navigation path SP from the position of the sensory unit 2until the spatialized sound source S reaches the end of thepredetermined second distance d2 and back to the position of the sensoryunit 2.

A Non-Limiting Example of Using the Auditory Cues is as Follows:

-   -   the first, the seventh, the eighth cues are simple sounds,    -   all the other cues are spatialized sounds.

The auditory feedback actuators 42 repeatedly output the spatializedsound source S by means of variation of the frequencies, duration,repetition, intensity, and 3D spatial virtualization directlyproportional to the predetermined preferred navigation path complexitycriteria.

The predetermined second distance d2 is inversely proportional to thepredetermined preferred navigation path SP complexity criteria, that isthe more complex the preferred navigation path SP is, the smaller thepredetermined second distance d2.

The predetermined second distance d2 typically varies between 0.2 m and5 m. If the preferred navigation path SP is very complex, thepredetermined second distance d2 typically varies between 0.2 and 1 m.The examples of the values for the predetermined second distance d2 aregiven for illustration purpose only and shall not be considered aslimiting.

Example: the predetermined second distance d2 is 1.2 m. This means thatthe spatialized sound source S is virtually travelling at 0.2 m from theposition of the sensory unit 2. The spatialized sounds travel back andforth from the position of the sensory unit 2 until they reach 1.2 m inthe direction of navigation and then they come back to the position ofthe sensory unit 2. As the speed of the sound is significantly higherthan the speed of human walk, the visually impaired user receives thenavigating guiding instructions in more detail than in any other guidingmode disclosed in this invention, because in the guiding mode using thevirtual travel of the spatialized sounds the sounds travel independentlyfrom the visually impaired user.

The features of the sounds, namely any between frequencies, duration,repetition, intensity, and 3D spatial virtualization or combinations ofthem, vary inversely proportional with the distance left until thepredetermined second distance d2. For example, the auditory cues aremore frequent and/or more intense and/or more 3D spatially virtualizedor last longer when the spatialized sound source S is at 0.1 m than whenthe spatialized sound source S is at 0.2 m.

The advantage of this guiding mode is that it allows a fine tuning ofthe navigation which makes it advantageous in environments where thewalkable area WA is very narrow and, consequently, the preferrednavigation path SP looks like a slalom between the objects On and theliving beings Ln.

In a second aspect of the invention, the wearable device 1 comprises theSensory unit 2, the Processing and control unit 3, the Feedback unit 4,the User commands interface 5. The wearable device 1 comprises twohardware units not represented graphically: the Power storage unit 6,and the memory M.

The term “memory M” shall be understood as designating a plurality ofnon-volatile memories either grouped together in a single distinctivehardware unit or spread in each of the other hardware units.

The memory M is configured to store at least the Live Map 310, all thealgorithms, all the criteria and requirements and the preferences of thevisually impaired user such as but not limited to the type of cues heprefers for receiving the guiding instructions. The storage is carriedout according to prior art.

The wearable device 1 is, in a preferred embodiment, a single-componentdevice, whereas in other preferred embodiments is a multi-componentdevice.

In case of the single-component device 1, all the hardware units areincluded in the wearable device 1 as shown in FIG. 2B. The wearabledevice 1 is mounted on the head, not represented graphically. Thepositioning of the hardware units on the head of the visually impaireduser can be, for example, in the form of a headband that has theadvantage of providing good support and anchor for all the hardwareunits.

In case of the preferred embodiments of the multi-component device 1,with reference to FIG. 2A FIG. 2A 1 and FIG. 2A 2, FIG. 6 and FIG. 7,one of the components is a headset component 11 comprising the Sensoryunit 2, and the Feedback unit 4. The headset component 11 can be, forexample in the form of a headband, as shown FIG. 2A 1, FIG. 2A 2 andFIG. 2B.

Two Non-Limiting Examples of the Preferred Embodiments of theMulti-Component Device 1 Depict Two Components:

-   -   the headset component 11, and    -   a belt-worn component 12, or, respectively, a wrist component        12. The wrist component 12 is not represented graphically.

In this case, the belt-worn component 12, or, respectively, the wristcomponent 12 comprises the processing and control unit 3, the Usercommands interface 5, and the power storage unit 6.

The memory M can be comprised in any of the two components or spreadamong them.

FIG. 2A 1 and FIG. 2A 2, depicts the preferred embodiment of thetwo-component device 1 having the headset component 11 and the belt-worncomponent 12.

The division of the components among the headset component 11 and thebelt-worn component 12, or, respectively, the wrist component 12 ismainly based on the size and weights of the units. The advantage ofusing the single-component device 1 is that its preferred location onthe head produces a sensorial experience for the visually impaired userof the wearable device 1 very close to the sensorial experience of thenon-visually impaired person, being close to the position of the earswhich enables hearing the auditory cues.

However, in some cases, some hardware units, such as the Processing andcontrol unit 3 and/or the Power storage unit 6 may be heavy and bulky.In these cases, the multiple-component device 1 has the advantage ofplacing the heavy and bulky hardware units in other locations of thebody such as but not limited to the belt or the wrist.

As the technology evolves in general towards miniaturization of hardwareunits, this will lead to increase the possibility of using thesingle-component device 1 without placing too much burden on the head ofthe visually impaired user.

In another preferred embodiment, not represented graphically, there arethree components:

-   -   The headset component 11 comprising the Sensory unit 2, the        Feedback unit 4,    -   The belt-worn component 12, or, respectively, the wrist        component 12 comprises the processing and control unit 3, the        power storage unit 6,    -   A hand-held component 13, not represented graphically,        comprising the User commands interface 5,

The memory M can be comprised in any of the headset component 11 or thebelt-worn component 12, or, respectively, the wrist component 12 orspread among the two.

The configuration of the various units composing the wearable device 1in order to work the invention is not influenced by the positioning ofsaid hardware units in the one- or, respectively multiple-componentdevice to the various parts of the human body.

The hardware units communicate between themselves either by wiredcommunication protocols or by wireless communication protocol, or by acombination of wired and wireless protocols, said communication takingplace according to prior art.

The Sensory Unit 2

The Sensory unit 2 has means configured to collect data regarding theenvironment of the visually impaired user.

The data collected by the Sensory unit 2 refers to multiplecharacteristics of objects On and living beings Ln that are generallyidentified by a human of good sensory capabilities including goodvision. The data, as collected by the Sensory unit 2, reflects thecomplexity of the environment with more accuracy than in the state ofart.

To satisfy the aim of collecting more accurate data, the Sensory unit 2requires a combination of sensors of multiple types that will bedescribed in detail. It shall be understood that all examples of sensorsare for a better understanding of the teaching of the invention andshall not limit the invention.

The Sensory unit 2 comprises four basic sensors: a Camera 21, a Depthsensor 22, a Inertial Measurement unit 23 and a Sound localisationsensor 24.

The best position of the Camera 21, the Depth sensor 22, and theInertial Measurement unit 23—irrespective of whether the wearable device1 is a single-component or a multi-component device, is on the foreheadas shown in FIG. 2A 1, FIG. 2A 2 and FIG. 2B. The reasons for thepreferred location the Sensory unit 2 on the forehead are threefold: i)because the human beings—visually impaired or not, in the absence of anydevice, are accustomed to move the head when receiving cues, such assounds or haptic cues, ii) because the forehead is not currently usedfor other devices or tasks, and iii) because the best field of view forthe Camera 21 and for the Depth Sensor 22 is on the forehead.

The configuration of the positioning of the Sensory unit 2 on theforehead of the visually impaired user must ensure that the field ofview 20 includes:

-   -   the feet of the visually impaired user,    -   the components of the free area A in the immediate proximity of        the feet,    -   the immediate steps of the visually impaired user,        The first sensor is the Camera 21. The term “Camera 21”        designates throughout the invention, one or several digital        video cameras. The invention requires to have at least digital        video camera.        The Camera 21 is configured to acquire 2 D images from a Camera        field of view, and to send the acquired 2D images to the        Localisation module 301, to the Walkable Area Detection module        302, and to the Object 2D Characteristics Extraction module 306.

The term “images” encompasses the static images as well as the videos,depending on the frame rate of acquisition of the images of the Camera21.

The images acquired by the Camera 21 refer to the visual characteristicsof the plurality of objects On and of the plurality of living beings Lnsuch as aspect; category—e.g. trees cars; colour, shape, dimensions aswell as the components of the free area A.

Non-limiting examples of Camera 21 include: HD Camera, having minimumvideo resolution 1280 pixels×720 pixels, VGA Camera, having minimumvideo resolution 320 pixels×240 pixels,

The Minimum Requirements of the Camera 21 are as Follows:

-   -   the horizontal field of view between at least 50° and up to        180°, the larger the better because it provides information from        a larger area, and    -   the vertical field of view between at least 60° and up to 180°        the larger the better because it provides information from a        larger area.

The Camera 21 can be RGB Camera or not. The RGB features help to providemore accurate information from the Camera field of view.

The more complex the Camera is, the more information will contain the 2D images acquired by the Camera.

The second sensor is the Depth sensor 22. The term “Depth sensor 22”designates throughout the invention one or several depth sensors. Theinvention requires to have at least one depth sensor.

The Depth sensor 22 is configured to acquire 3D point clouds datacorresponding to 3D distance position and dimension for each of theobjects On and each of the living beings Ln placed in the Depth sensorfield of view as a continuous point cloud, and to send them

to the Localisation module 301, to the Walkable Area Detection module302, and to the Object 3D Characteristics Fusion module 307.

The 3D point cloud data acquired by the Depth sensor 22 refers to the3-D physical characteristics of the objects On and the living beings Lnsuch as density, volume, etc.

Non-limiting examples of Depth sensor 22 are stereoscopic camera, radar,Lidar, ultrasonic sensor, mmWave radar sensor. Using mmWave radar sensoris advantageous because it is able to sense the pulse or the breath ofthe living beings Ln, even when the living beings Ln are moving whichbrings additional information for the visually impaired user.

It is possible to combine the Camera 21 and the Depth Sensor 22 in asingle sensor Camera and Depth Sensor 21-22. The advantage is reducingthe size and weight of the two afore-mentioned sensors by using only onesensor configured to carry out the tasks of the two sensors. Onenon-limiting example of Camera and Depth Sensor 21-22 would be a time offlight TOF camera.

The third sensor is the Inertial Measurement unit 23. The term “InertialMeasurement unit 23” designates throughout the invention an ensemblemade of at least one accelerometer and at least one gyroscope and,either as separate sensors, or combined sensors. It is preferable to addat least one magnetometer for better accuracy, either as a separatesensor or combining it with the at least accelerometer and/or the atleast gyroscope. It is better to use combined sensors because of theneed to reduce the size and weight of the ensemble. The inventionrequires to have at least one inertial measurement unit.

The Inertial Measurement unit 23 is configured to determine theorientation of the Sensory unit 2, and to send the determinedorientation to the Localisation module 301, and to the CharacteristicsFusion module 307 by means of the Orientation Computation module 303.

Since the Sensory unit 2 is placed on the forehead of the visuallyimpaired user, the information acquired by the Inertial Measurement unit23 implicitly refers to orientation of the head of the visually impaireduser in respect to the ground.

The fourth sensor is the Sound localisation sensor 24.

The term “Sound localisation sensor 24” designates throughout theinvention an ensemble of one or several sensors used to determine thesource of various sounds in the three-dimensional space usually by thedirection of the incoming sound waves and the distance between thesource and sensor(s).

The Sound localisation sensor 24 is configured to acquire a plurality ofsound streams in the three-dimensional space emitted by the objects Onand the living beings Ln, and to send them to the Sound DirectionLocalisation module 304.

The information acquired by the Sound localisation sensor 24 refers tothe sounds emitted by the objects On and the living beings Ln, includingthe directionality of said sounds.

The coverage of the environment by the Sound localisation sensor 24 isdefined by its beam pattern.

A non-limiting example of sound localisation sensor is a microphonearray. The minimum number of microphone arrays used for the Soundlocalisation sensor 24 must be such that the sum of the beam patternequals to the angle of the field of view 20. The maximum number ofmicrophone arrays used for the Sound localisation sensor 24 covers 360°.The microphone arrays are positioned within the headset such that thesum of their beam pattern be comprised between the angle of the field ofview 20 and 360°.

The basic sensors receive from the Sensory fusion sub-unit 30 of theProcessing and control unit 3 specific configurations, including thecorrelation of the respective field of views of the Camera 21, Depthsensor 22, with the range of measurement of the Inertial Measurementunit 23 and the beam pattern of the Sound localisation sensor 24.

Said correlation has as result the field of view of the basic sensors20, depicted schematically in FIG. 3A and FIG. 3B. The concept of thefield of view of the basic sensors 20 does not mean all basic sensorshave exactly the same range. It should be understood as the area whereall the basic sensors have perception, similarly with the concept of thecommon denominator in mathematics. Typically, the field of view of thebasic sensors 20 faces forward.

However, the Sound localisation sensor 24 may have a wider range thatthe field of view of the basic sensors 20, for example when the numberof microphone arrays is such that the sum of the beam pattern equals to360°. This is advantageous because it allows gathering sound informationoriginating from the back of the visually impaired user.

In another preferred embodiment, depicted in FIG. 13, it is possible toprovide more information about the environment of the visually impaireduser by adding one or both additional sensors: a Global positioningsensor 25, and a Temperature sensor 26.

Any combination of each of the additional sensors with the group ofbasic sensors has the advantage of providing additional information tothe Processing and control unit 3 which leads to a more accurate anddetailed Live Map 310.

Each of the two additional sensors has a corresponding module in thesensory fusion sub-unit 30, as follows:

The Global positioning sensor 25 is configured to determine the absoluteposition of the Sensory unit 2 and to send the determination to aRelative to Absolute Conversion module 309-1 that converts the relativeposition of the Sensory Unit 2 into absolute position, thus the positionof the objects On and the position of the living beings Ln is expressedas absolute position.

The best position of the Global positioning sensor 25 is on the top ofthe headset component 11 of the wearable device 1 in case ofmulti-component device, respectively on the top of the wearable device 1in case of single component device.

In the absence of the Global positioning sensor 25, the Sensory Fusionsub-unit 30 determines the relative position of the wearable device 1 inrespect to each of the objects On and to each of the living beings Ln.

The Temperature sensor 26 is configured to determine the temperature ofthe objects On and of the living beings Ln, and to send the determinedtemperature to an Object Temperature Characteristics fusion module309-2.

In case of using either of the additional sensors, the data outputted bythe Object Sound Characteristics Fusion module 308 is sent to either theRelative to Absolute Conversion module 309-1 or the Object TemperatureCharacteristics fusion module 309-2 respectively, fused with the datasent by the respective sensor and the outcome is sent to the Live Mapsub-unit 31.

In case of using both additional sensors, as depicted in FIG. 13, thedata outputted by the Object Sound Characteristics Fusion module 308 issent to the Relative to Absolute Conversion module 309-1, fused with thedata from the Global positioning sensor 25, then the outcome is sent tothe Object Temperature Characteristics fusion module 309-2 fused withthe data sent by the Temperature sensor 26 and the outcome is sent tothe Live Map sub-unit 31.

The Processing and Control Unit 3

The Processing and control unit 3 is a computing unit, comprising atleast one processor and at least one non-volatile memory, such as butnot limited to a microcontroller, a computer, a supercomputer. The term“computing unit” encompasses a single computing unit or a plurality ofcomputing units located remotely from one another communicating within acomputer communication system.

The Processing and control unit 3 comprises: the Sensory fusion sub-unit30, the Live Map sub-unit 31, the Relationship Manager sub-unit 32, theNavigation Manager sub-unit 33, the User commands interface Managersub-unit 34, the Feedback Manager sub-unit 35, and the Soundrepresentation sub-unit 36.

With reference to FIG. 5, the Sensory fusion sub-unit 30 comprises meansconfigured to fuse and correlate the determinations received from thefour basic sensors of the Sensory unit 2.

With reference to FIG. 13, the Sensory fusion sub-unit 30 comprisesmeans configured to fuse and correlate the determinations received fromthe four basic sensors of the Sensory unit 2 and from either or bothadditional sensors.

The Localisation module 301 comprises means configured to localize thecurrent position and orientation of the sensory unit 2 of the wearabledevice 1 and of the plurality of the objects On and living beings Ln inrespect to the sensory unit 2, in 3D coordinates, on the current LiveMap 310 by means of localisation algorithms applied to the data acquiredfrom the Camera 21, the Depth sensor 22, the Inertial Measurement unit23 of the Sensory unit 2.

The Localisation module 301 further comprises means configured to sendthe localisation of the position and orientation of the sensory unit 2to the Walkable Area Detection module 302, thus updating the Live Map310 content as outputted from S2.2 with the layer referring to thelocalisation data of the position and orientation of the sensory unit 2in respect to the plurality of the objects On, and, respectively to theplurality of living beings Ln.

The Walkable Area Detection module 302 comprises means configured toreceive the data acquired from the Camera 21, the Depth sensor 22, andmeans configured to receive data from the Localisation module 301, and,based on both sources of data, means configured to define the walkablearea WA, and the conditional walkable area CWA, and send them to theLive Map sub-unit 31, by applying the set of permanent predeterminedwalkable area requirements and predictable conditional walkable arearequirements, stored in the memory M.

The Orientation Computation module 303 comprises means configured todetermine the orientation of the wearable device 1 based on the inertialdata provided by the Inertial Measurement unit 23, and to sends thedeterminations to Object 3D Characteristics Fusion module 307.

The Sound Direction Localisation module 304 comprises means configuredto determine the direction of the plurality of sound streams expressedin 3D coordinates emitted respectively by each of the plurality of theobjects On and the plurality of living beings Ln based on the datareceived from the Sound localisation sensor 24 and means configured tosend the determined direction to the Sound Classification module 305.

The Sound Classification module 305 comprises means configured toclassify into sound types the plurality of sound streams received fromthe Sound Direction Localisation module 304 and to send the classifiedsound types to the Object Sound Characteristics Fusion module 308. Themeans configured to classify into sound types the plurality of soundstreams typically use artificial intelligence algorithms.

The Object 2D Characteristics Extraction module 306 comprises meansconfigured to provide the pixel-wise segmentation of the 2D imagesacquired from the Camera 21, to detect in the pixel-wise segmented 2Dimages each object On of the plurality of the objects On, and eachliving being Ln of the plurality of living beings Ln placed in the fieldof view 20, to determine their respective position in 2D coordinates,and their respective physical characteristics and to send thedeterminations to the Object 3D Characteristics Fusion module 307.

The Object 3D Characteristics Fusion module 307 comprises meansconfigured to receive data from the Object 2D Characteristics Extractionmodule 306, from the Orientation Computation module 303 and from theDepth sensor 22, and to determine:

-   -   the position in 3D coordinates of each of the objects On in        respect to the Sensory unit 2, and their orientation in respect        to the Sensory unit 2, and the future position at predetermined        moments in time based on the vector of movements, respectively.    -   the physical characteristics of the plurality of the objects On,        such as dimensions, composition, structure, colour, shape,        humidity, temperature, degree of occupancy, degree of        cleanliness, degree of usage, degree of wear, degree of        stability, degree of fullness, degree of danger,    -   the position of each of the living beings Ln in 3D coordinates,        their physical characteristics, like height, and skeleton pose        orientation, as well as the prediction of their future position        in the predetermined unit of time based on the vector of        movement,    -   the current activity and mood status of each of the living        beings Ln based on skeleton pose orientation, and their physical        characteristics.

The Object Sound Characteristics Fusion module 308 comprises meansconfigured to add acoustical characteristics to each of the plurality ofthe objects On and the living beings Ln for which the 3D coordinateshave been determined based on the classified sound streams typesdetermined by the Sound Classification module 305 by associating withthe detected objects On and the living beings Ln and send all data tothe Live Map sub-unit 31.

In an embodiment of the present invention, the Sensory fusion sub-unit30 further comprises the Relative to Absolute Conversion module 309-1.This module comprises means configured to convert the relative positionof the Sensory Unit 2 into absolute position, to fuse the data from theObject Sound Characteristics Fusion module 308 with the data regardingabsolute position of the Sensory unit 2 and to send the determinationsto the Live Map sub-unit 31 either directly or by means of the ObjectTemperature Characteristics fusion module 309-2.

In another embodiment of the present invention, the Sensory fusionsub-unit 30 further comprises the Object Temperature Characteristicsfusion module 309-2. This module comprises means configured to determinethe temperature of the detected objects On and the living beings Ln, tofuse the data from the Object Sound Characteristics Fusion module 308with the data regarding the temperature of the objects On and of theliving beings Ln and to send the fused data to the Live map sub-unit 31.If the Relative to Absolute Conversion module 309-1 is used, it sendsthe data to the Object Temperature Characteristics fusion module 309-2and finally fuses the data with the data regarding the temperature ofthe objects On and of the living beings Ln.

According to the invention, the Live Map sub-unit 31 comprises meansconfigured to create, repeatedly update and store the Live Map 310 andmeans to receive data referring the components of the free area A andthe updated Live Map 310 content as outputted from S2.2 with the layerreferring to the localisation of the position and orientation of thesensory unit 2 from the Walkable Area Detection module 302, dataregarding each of the plurality of the objects On and the living beingsLn in 3D coordinates including acoustical characteristics from theObject Sound Characteristics Fusion module 308, and to send all Live mapdeterminations to the Localisation module 301.

The Live Map sub-unit 31 comprises means configured to receive:

-   -   The queries of the Live Map 310 by the Relationship Manager        sub-unit 32,    -   The queries of the Live Map 310 by the Navigation Manager        sub-unit 33, including the query of the User commands interface        Manager sub-unit 34 to the Navigation Manager sub-unit 33 if the        point of interest PI is already in the Live Map 310,    -   The updated relationships Rn carried out by the Relationship        Manager sub-unit 32    -   The updated components of the free area A carried out by the        Navigation Manager sub-unit 33,    -   The queries of the Sound representation module 36 regarding the        specific information regarding the Objects On.        The Live Map Sub-Unit 31 Comprises Means Configured to Send:    -   the plurality of relations Rn in response to the queries of the        Live Map 310 by the Relationship Manager sub-unit 32,    -   the components of the free area A in response to the queries of        the Navigation Manager sub-unit 33,    -   all Live map determinations to Navigation Manager sub-unit 33 in        response to the queries of it, The Relationship Manager sub-unit        32 comprises means configured to query the Live Map 310 and to        import the most recently updated data from the Live Map 310 as a        result of querying. Said most recently updated data refers to:    -   the plurality of objects On,    -   the plurality of living beings Ln,    -   the conditional walkable area CWA, because the some of the        objects On and/or some of the living beings Ln are related to        said conditional walkable area CWA e.g. the traffic light, or        the dog,    -   the existing relationships Rn prior to the query.

Further on, the Relationship Manager sub-unit 32 comprises meansconfigured to carry out computations for determining and updating therelations between the plurality of objects On and/or the plurality ofliving beings Ln, and to send the updated relations as result of thecomputations to the Live map sub-unit 31 to store same in the Live Map310.

The Navigation Manager sub-unit 33 comprises means configured to:

-   -   determine, repeatedly update and store in the memory M, of at        least one navigation path Pn,    -   repeatedly select the preferred navigation path SP from the at        least one navigation path Pn,    -   repeatedly send the preferred navigation path SP, together with        the associated navigation guiding instructions, to the Feedback        Manager sub-unit 35.    -   receive from the User commands interface Manager sub-unit 34 the        initiation requests, the selection requests and the information        requests.    -   query the Live Map 310 for the Live Map determinations based on        fused data received from the Sensory Fusion sub-unit 30 and the        components of the free area A: walkable area WA, conditional        walkable area CWA and non-walkable area NA and to receive from        the Live Map sub-unit 31 the response corresponding to each        query.    -   receive the query of the User commands interface Manager        sub-unit 34 if the point of interest PI is already in the Live        Map 310,    -   verify if the at least two navigation path requirements are met.    -   send to the Feedback Manager sub-unit 35 the associated        navigation guiding instruction associated to said at least one        predictable conditional walkable area requirement.

The User commands interface Manager sub-unit 34 comprises meansconfigured to receive requests and selections that the visually impaireduser makes by means of the User commands interface 5 and to transmitthem to the Navigation Manager sub-unit 33 and means configured to sendselected guiding modes to The Feedback Manager sub-unit 35.

The User commands interface Manager sub-unit 34 further comprises meansfor receiving requests from the visually impaired user for soundrepresentation of a specific category of objects On or a specificcategory of living beings Ln from the Live Map 310.

The Feedback Manager sub-unit 35 comprises means configured to guide thevisually impaired person along the preferred navigation path SP byreceiving the guiding instructions from the Navigation Manager sub-unit33 together with selected guiding modes from the User commands interfaceManager sub-unit 34 and means configured to transmit the correspondingassociated guiding instructions to the Feedback unit 4, and furthercomprises means for sending the sound representation regarding aspecific category of objects On or a specific category of living beingsLn.

In the embodiments where the Sound representation sub-unit 36 is aself-contained sub-unit and a sub-unit of the Navigation Managersub-unit 33, the Feedback Manager sub-unit 35 further comprises meansfor receiving sound representation of the specific category of objectsOn or a specific category of living beings Ln from the Soundrepresentation sub-unit 36.

The Sound representation sub-unit 36 comprises means configured toreceive requests from the visually impaired and to extract from the LiveMap 310 of the corresponding information regarding a specific categoryof objects On or a specific category of living beings Ln and means forrepresenting the extracted information as corresponding spatializedsounds and transmitting same to the Feedback Unit 4.

The Feedback unit 4, configured to be placed on the head of the visuallyimpaired user, comprises means configured to guide the visually impaireduser along the preferred navigation path SP by receiving the associatedguiding instructions from the Feedback Manager sub-unit 35 and bysending the haptic and/or auditory cues to the visually impaired personas it was described in detail in the section regarding the details ofthe step 4 of the method, and comprises means for sending to thevisually impaired user the sound representation the specific category ofobjects On or a specific category of living beings Ln.

The User commands interface 5, configured to be placed on the head ofthe visually impaired user, comprises means configured to receive fromthe visually impaired user the requests, namely the initiation request,the selection request and the information request and the selections ofthe guiding modes and to send them to the User commands interfaceManager sub-unit 34.

Non Limiting Examples of the User Commands Interface 5 are as Follows:

-   -   User commands haptic means 51, e.g., buttons used for simple        requests corresponding to frequent predetermined points of        interest PI. For example, a first button can be named “home”        corresponding to the door of the entrance to the home where the        visually impaired person lives, a second button can be named        “bathroom”, a third button can be named “kitchen”, etc. The        buttons can be analogues or digital.    -   User commands audio means 52, e.g., microphones for the points        of interest PI that are not frequent. The user commands audio        means include speech recognition means and means to transform        the words of the visually impaired user into instructions sent        to the User commands interface Manager sub-unit 34. Taking the        same example with the room with four windows, the visually        impaired person says “window” to the microphones 52 and all four        windows of the room are represented in sounds.

The communication of the User commands interface 5 with the visuallyimpaired person and with User commands interface Manager sub-unit 34 isaccording to prior art.

The Power Storage Unit 6

The term “Power storage unit 6” shall be understood as designating oneor several batteries configured to power the other hardware units of thewearable device 1. The way the Power storage unit 6 powers said otherhardware units of the wearable device 1 is carried out according toprior art.

The Communication unit 7 comprises means configured to download mapsfrom the Internet, such as but not limited to the downloadable maps.

In a third aspect of the invention, it is provided a computer programcomprising instructions which, when the program is executed by thewearable device 1 causes the wearable device 1 to carry out the steps ofthe computer-implemented method for assisting the movement of a visuallyimpaired user, in any of the preferred embodiments, includingcombinations thereof.

In a fourth aspect of the invention, it is provided a computer readablemedium having stored thereon instructions which, when executed by thewearable device 1, causes the wearable device 1 to carry out the stepsof the computer-implemented method, in any of the preferred embodiments,including combinations thereof.

In a fifth aspect of the invention, it is provided a non-transitorycomputer-readable storage device storing software comprisinginstructions executable by one or more computers which, upon suchexecution, cause the one or more computers to perform operations of thecomputer-implemented method, in any of the preferred embodiments,including combinations thereof.

In a sixth aspect of the invention, it is provided a system comprisingone or more computers and one or more storage devices storinginstructions that are operable, when executed by the one or morecomputers, to cause the one or more computers to perform operations ofthe computer-implemented method, in any of the preferred embodiments,including combinations thereof.

The terms “computers” of the fifth and sixth aspects refer to acomputing unit, comprising at least one processor and at least onenon-volatile memory, such as but not limited to a microcontroller, acomputer, a supercomputer. The term “computing unit” encompasses asingle computing unit or a plurality of computing units located remotelyfrom one another communicating within a computer communication system.

Example No. 1

The detailed description of the method is exemplified in a real-lifescenario, with reference to the FIGS. 14, 15, 16, and 17. The personskilled in the art shall understand that the teaching of the inventionis not limited to this example.

In the real-life scenario, the visually impaired person 1 is on thesidewalk of a street in the close proximity of the entrance to abuilding. He wants to get into the building thus has to navigate fromhis standpoint until the entrance door of the building and has also tofind the doorbell of the entrance door.

This is a non-limiting example when the visually impaired user sends theinitiation request in order to be guided to the entrance door of thebuilding.

In FIG. 14 and FIG. 15 it is Shown a Portion of the Live Map 310 thatIncludes:

-   -   A building with its parts: each step, the fences, each of the        two handrails, an entrance door 84 considered as an initial        point of interest, with component parts such as a door lock, a        door knob and a door bell 841, the doorbell 841 being considered        a further point of interest.    -   a dog 83 recognized as living being Ln.    -   the walkable area WA 942, that includes the sidewalks and the        stairs to the entrance in the building.    -   the non-walkable area NA 941, that includes the street.    -   the conditional walkable area 943: two pedestrian crossings 832        each one provided with a corresponding traffic light 831.

An example of the geometric predetermined walkable area requirementsinclude: the height of the sidewalk must not exceed 7 cm, the distanceto the fences must not exceed 0.5 m, the distance to the margins of thesidewalk must not exceed 0.5 m, the height of the virtual cuboid is 2.20m, that is 40 cm more than the height of the visually impaired personthat is 1.80 m.

An example of static and dynamical physical relationships Rn is therelation created in the live map 310 by the Relationship Managersub-unit 32 of the Processing and control unit 3 with respect toassociating the colour of the traffic lights 831 to the conditionalstatus of the conditional walkable area 943: if the colour is green, thearea 943 is walkable whereas if the colour is red, the area 943 isnon-walkable.

A non-limiting example for the conditional walkable area CWA isrepresented by the two pedestrian crossings 832 provided with trafficlights 831. The streets are defined as non-walkable area NA in thepermanent predetermined walkable area requirements. When it comes to thepedestrian crossings 832, in case there are no traffic lights, they arepredefined as walkable area 942, whereas in case there are trafficlights, they are predefined as conditional walkable area 943 that isthey are walkable only when the colour of the traffic lights 831 isgreen. This is an example of at least one predictable conditionalwalkable area requirement, as colour of the traffic lights changespredictably changing from red to green and from green to red.

The visually impaired user 1 is on the sidewalk of the building when hesends the initiation request. In several embodiments of the invention,the entrance door 84 is already in the Live Map 310 because it was addedto it in the step 2 of the method in the past.

In the embodiment of the invention where said entrance door 84 is notyet in the Live Map 310 at the moment of sending the initiation requestbecause the visually impaired user has just got off from a taxi to acompletely new place and consequently the entrance door 84 was neveradded before to the Live Map 310, the Navigation Manager sub-unit 33determines the wandering path WP to repeatedly refocus the field of view20, while S1 and S2 of the method are repeated until said entrance door84 is found and stored in the Live Map 310.

In the embodiment of the invention where the entrance door 84 is notknown by the visually impaired user, because the visually impaired userhas just got off from a taxi to a completely new place where are twoentrance doors 84-01 and 84-02 one close to another, the visuallyimpaired user sends an information request to the Navigation Managersub-unit 33 for finding “entrance door”. Then the Navigation Managersub-unit 33 queries the live Map 310 for the entrance doors in the areaof interest from the proximity of the visually impaired user and findsthat there are two entrance doors 84-01 and respectively 84-02.

If the two entrance doors 84-01 and 84-02 are not already stored in theLive Map 310, the Navigation Manager sub-unit 33 determines thewandering path WP until said entrance doors 84-01 and 84-02 are foundand stored in the Live Map 310.

Once the two entrance doors 84-01 and 84-02 are found and stored in theLive Map 310 the Navigation Manager sub-unit 33 represents each of themas corresponding spatialized sounds and transmits same to the FeedbackUnit 4 via the Feedback Manager sub-unit 35. Then the visually impaireduser selects one among the entrance doors 84-01 and 84-02 as theentrance door 84 that constitutes his initial point of interest.

The Navigation Manager sub-unit 33 determines in S3 a single navigationpath Pn, namely, an initial navigation path 911 for the visuallyimpaired user to navigate from his standpoint t₀ the entrance door 84.The preferred navigation path SP is thus the initial navigation path911. When the visually impaired user 1 navigates along the initialnavigation path 911, the dog 83 is sensed by the Sensory unit 2.

The aggressivity of the dog is sensed as follows:

-   -   if the dog barks, this is sensed by the Object Sound        Characteristics Fusion module 308,    -   if the dog has an aggressive expression on its face, this is        sensed by the Object 2D Characteristics Extraction module 306,    -   if the dog is moving or is trembling because it is furious, this        is sensed by the Object 3D Characteristics Fusion module 307.

Since the data sensed by the basic sensors and, where applicable, by theadditional sensors is fused and then sent to the Live Map sub-unit 31such that to be included in the Live Map 310, the Navigation Managersub-unit 33, when querying the Live Map 310, checks the at least twonavigation path requirements and detects that the non-aggressivityrequirement is not met. For this reason, the Navigation Manager sub-unit33 it determines a secondary navigation path 912 towards the sameinitial point of interest PI 84. The preferred navigation path SP is nowthe secondary navigation path 912, which avoids the dog 83 having anadverse reaction.

With reference to FIGS. 14, 15, and 16, the secondary path 912 mustcross a road, which is an example of non-walkable area NA, using the twopedestrian crossings 832 which are examples of conditional walking areaCWA 943.

When the visually impaired user 1 approaches the first pedestriancrossing 832, the Relationship Manager sub-unit 32 determine that theconditional area 943 is conditioned by the colour of the first trafficlight 831.

Therefore, a conditional relation is built in the Live Map 310, by theRelationship Manager sub-unit 32, relating the colour of the firsttraffic light 831 to the conditional status of the first pedestriancrossing 832.

When the traffic light 831 turns green, the conditional walkable area943 is considered walkable and the visually impaired user 1 receives theassociated navigation guiding instruction to continue the navigation onthe secondary path 912.

The same repeats on the second pedestrian crossing 832.

FIG. 16 depicts the visually impaired user 1 navigating on the secondarypath 912 waiting at the second pedestrian crossing 832 for the colour ofthe second traffic light 831 to turn to green. With reference to FIG.17, the visually impaired user has already crossed both pedestriancrossings 832 and is approaching the initial point of interest namelythe entrance door 84. The Relationship Manager sub-unit 32 determines anew parent-child relation between the initial point of interest, namelythe door 84 and the further point of interest, namely the doorbell 841.Therefore, a new item 841 is created in the Live Map 310 correspondingto the doorbell 841 and the new parent-child relation between the thedoor 84 and the doorbell 841 is created by Relationship Manager sub-unit32 and updated in the Live Map 310. The doorbell 841 replaced the door84 as point of interest PI.

In FIG. 15 and in FIG. 16 it is depicted a walkable tunnel 922 that hasa cross-section of around 1 m, that is around 0.5 m to the left andaround 0.5 m to the right of the secondary navigation path 912, thelatter being represented as a line.

If the preferred navigation path SP passes through an indoor space, suchas an apartment, the cross-section is usually smaller, for examplearound 0.5 that is around 0.25 m to the left and around 0.25 m to theright of said preferred navigation path SP.

The details of the guiding of the visually impaired user through thewalkable tunnel 922 are exemplified below in relation to FIG. 15 andFIG. 16. In this example, the wearable device 1 is provided with theleft haptic feedback actuators 411, the right haptic feedback actuators412 and the centre haptic feedback actuators 413, each of themcomprising one or more linear resonant actuators.

In this example, the three-dimensional walkable tunnel T is selected forreceiving the associated navigation guiding instructions.

The visually impaired user receives the start command by the firsthaptic cue—which is temporal, and the visually impaired user beginsnavigating.

The Feedback Manager sub-unit 35 will attempt to keep the visuallyimpaired user on the preferred navigation path SP and within the limitsof the walkable tunnel 922 by giving directional haptic cues.

If the visually impaired user, when navigating, is too close to the leftside of the walkable tunnel 922, the second haptic cue—which isspatiotemporal, is received by the left feedback actuators 411. Thelinear resonant actuators of the left feedback actuators 411 outputvibrations in rapid succession, one linear resonant actuator vibratingafter another, in the direction in which the visually impaired user mustreorient, that is to the right, giving the visually impaired user thetactile sensation of having the forehead dragged by someone to theright. The variation of the duration, periodicity, intensity orfrequency of the vibration of the second haptic cue is proportional tothe degree of closeness to the left side of the the walkable tunnel 922.

If the visually impaired user, when navigating, is too close to theright side of the walkable tunnel 922, the third haptic cue is receivedby the right feedback actuators 412—which is spatiotemporal, havingidentical configuration with the one of the second haptic cue exceptthat it indicates as direction of reorientation the left instead of theright. The variation of the duration, periodicity, intensity orfrequency of the vibration of the third haptic cue is proportional tothe degree of closeness to the right side of the the walkable tunnel922.

Guiding the user forwards is by the fourth haptic cue, —which isspatiotemporal. The fourth haptic cue is received by the centre feedbackactuators 413. The variation of the duration, periodicity, intensity orfrequency of the vibration of the fourth haptic cue is proportional tothe speed that the visually impaired user should have when navigating.

If the visually impaired user, when navigating, must reorient hisdirection of movement, on the horizontal plane, for example turn rightwhen he arrives to the pedestrian crossroad 943 shown in FIG. 16, thefifth haptic pattern cue is received—which is spatiotemporal, by thecentre feedback actuators 413. The variation of the duration,periodicity, intensity or frequency of the vibration of the fifth hapticcue is proportional to the degree of the turn.

If the visually impaired user, when navigating, must reorient hisdirection of movement on the vertical plane, for example when thevisually impaired user has already crossed the pedestrian road 943 andis approaching the stairs of the building and has to climb some stairs,the sixth haptic cue is received—which is spatiotemporal, by the centrefeedback actuators 413. The variation of the duration, periodicity,intensity or frequency of the vibration of the sixth haptic cue isproportional to the amount of movement required to the visually impaireduser.

When the visually impaired user, arrives to the pedestrian crossroad 832shown in FIG. 15 and FIG. 16, if the color of the traffic road 831 isred, the seventh haptic cue is received—which is temporal, correspondingto the associated navigation guiding instruction to temporary stop bythe centre feedback actuators 413. Then, when the the traffic light 831turns green, the seventh haptic cue is received again, this timecorresponding to resuming the navigation.

The eighth haptic pattern cue—which is temporal, signals the end of thenavigation as the point of interest PI is reached, being received fromthe centre feedback actuators 413.

Further types of haptic pattern cues can be defined to accommodatenavigation situations or requirements of the user. For example, if thevisually impaired user, when navigating, is centered within the walkabletunnel 922 of the secondary navigation path 912, the right feedbackhaptic actuators 412 and the left feedback haptic actuators 411 caneither not present any type of haptic pattern cues, or present a ninthtype of haptic pattern cue on both sides of the forehead, to signal thevisually impaired user that he is navigating centered within thewalkable tunnel 922.

Group of Examples No. 2

Taking the example from the description when the visually impaired userenters a new room that has four windows 85, the first 85-1, the secondwindow 85-2, the third window 85-3, and the fourth window 85-4, and hewants to open one of the four windows 85, the potential point ofinterest PPI is the group of the four windows as at least one object Onselected from the plurality of objects On.

The term “potential” signifies that any of the windows 85 of the roommay be selected as initial point of interest PI.

With reference to FIGS. 18 to 28, the visually impaired user sends insub-step S.3-0.1 the information request named “window” as potentialpoint of interest PPI because that he is interested to find out moredetails regarding the windows from the room in order to choose theinitial point of interest PI.

The person skilled in the art shall understand that the examplesdescribed apply to any kind of Objects On, and mutatis mutandis to thecategories of living beings Ln.

In sub-step S.3.-0.2 the Sound representation sub-unit 36 representseach of the four windows 85, as corresponding spatialized sounds: thefirst spatialized sound S86-1, the second spatialized sound S86-2, thethird spatialized sound S86-3, and the spatialized sound fourth S86-4and transmits the four spatialized sounds to the Feedback Unit 4 viaFeedback Manager sub-unit 35, when the Sound representation sub-unit 36is not part of said Feedback Manager sub-unit 35.

In sub-step S.3-0.3 the visually impaired user selects as initial pointof interest PI one from the four windows 85-1, 85-2, 85-3, 85-4, andtransmits the corresponding selection request just like any otherselection request.

Representation in sounds of the sub-step S.3-0.2 is exemplified belowwith reference to the FIGS. 18 to 28 for a single window, or for twowindows from the group of four windows 85-1, 85-2, 85-3, 85-4. Theperson skilled in the art shall understand that the teaching of theinvention is not limited to this group of examples and that thereference to the window 85 stands for any of the four windows 85-1,85-2, 85-3, and 85-4. Likewise, the reference to the spatialized soundS86 stands for each of the corresponding spatialized sounds S86-1,S86-2, S86-3, and S86-4.

In example 2-1 with reference to FIG. 18, in a preferred embodiment, theSound representation module 36 extracts from the Live map 310 thespecific information regarding the window 85 which is the potentialpoint of interest PPI, encodes them into the spatialized sound S86perceived by the visually impaired user as emitted from the location ofthe window 85, and sends the spatialized sound S86 to the visuallyimpaired user by means of auditory Feedback actuators 42. In order toillustrate additional features of the window 85 representing saidpotential point of interest PPI, such as its dimensions, or density ofthe material, the Navigation Manager 33 will further encode thespatialized sound S86 into a spatialized sound having a particularfrequency 586 f, a particular time period S86 t and/or a particularpulse S86 p, not represented graphically, said frequency 586 f,particular time period S86 t and/or pulse S86 p corresponding to saidadditional features.

In examples 2-2 and 2-3 with reference to FIG. 19A and, respectivelyFIG. 19B, in another preferred embodiment, the Sound representationsub-unit 36 encodes the specific information regarding the selectedwindow 85 into the spatialized sound S86, that has an exaggerated or“zoomed” representation in respect to the location of the potentialpoint of interest 85. If, for example, the potential point of interest85 is at a range of 10 meters, it will be represented in the spatializedsound S861 at a range of, for example, 1 meter. This zoom orexaggeration effect is presented to the visually impaired user on boththe elevation as shown in FIG. 19A, and on the azimuth, as shown in FIG.19B.

In example 2-4, with reference to FIG. 20, in another preferredembodiment, the visually impaired user sends the request for Soundrepresentation of two windows 85-1, and 85-2 that are placed atnon-equal distances from the visually impaired user.

The Sound representation sub-unit 36 encodes the specific information ofthe selected windows 85-1, and 85-2 from the Live map 310 into thespatialized sounds S86 f-1, and 586 f-2 having different frequencyfeatures depending on the distance of the windows 85-1, and 85-2 to thevisually impaired user, and sends the encoded spatialized sounds S86f-1, and 586 f-2 to the visually impaired user.

Thus, for example, the corresponding audio cues of the spatializedsounds S86 f-1, and 586 f-2 corresponding to the additional features ofthe window 85 sent to the visually impaired user vary in frequencies:the cues last longer and/or the degree of repetition is higher for thewindow 85-2 than the one that is nearer to the visually impaired user,85-1 respectively.

In example 2-5, with reference to FIG. 21, in another preferredembodiment, the visually impaired user sends the request to represent insounds the margins of the window 85. For simplicity, it was chosen torepresent only two of the extremities, respectively 85-E1 and 85-E2, theperson skilled in the art understands that the same representationapplies for all extremities of the window, depending on its shape.

The Sound representation sub-unit 36 extracts the specific informationof the selected window 85 from the Live map 310, encodes it intospatialized sounds S86P-E1, and S86P-E2 corresponding to the windowextremities 85-1E and 85-E2, the spatialized sounds S86P-E1, and S86P-E2having different encoding characteristics depending on the distance ofeach of the two chosen extremities relative to the visually impaireduser. The distance can be measured either on the azimuth, on theelevation or on the range of the window 85, or in any combination of theaforementioned.

In example 2-6, with reference to FIG. 22, in another preferredembodiment, the visually impaired user sends the request by the Usercommands interface 5 to the Sound representation sub-unit 36 via theUser Commands Interface sub-unit 34 for representation of the dimensionsof the window 85 as a potential point of interest PPI.

The Sound representation sub-unit 36 encodes the specific information ofthe dimensions of the selected window 85 extracted from the Live Map 310into temporal spatialized sound S86P representing punctiform soundsalong one of the three spatial dimensions between chosen extremities ofthe window 85 or a linear sound S86L moving on a straight-line path fromthe extremity 85-E1 to the extremity 85-E2, and sends them to thevisually impaired user by means of auditory Feedback actuators 42. Thesame operation is carried out for the others extremities of the window85, specifically 85-E3, and 85-E4 in case the window 85 is rectangular(not represented graphically).

The dimensions of the window 85 are measured between the extremities85-E1, and 85-E2, 85-E3, and 85-E4 of the window 85 along thecorresponding spatial dimensions by means known from the prior art.

In examples from 2-7 to 2-10 with reference to FIGS. 23-26, in anotherpreferred embodiments, the visually impaired user sends the request forrepresentation of the shape of the window 85 or a window part. In theexamples from 2-7 to 2-10, the window 85 can be a decorative window thatis not rectangular, or a mirror or a decorative part of a door. Forsimplicity it will be referenced as window 85.

The Sound representation sub-unit 36 extracts the specific informationfrom the Live map 310, encodes it into temporal spatialized sounds S86representing the shape of the window 85.

In example 2-7, with reference to FIG. 23, the shape of a window 85 isrepresented by two spatialized punctiform sounds S86P1, and S86P2, eachof them virtually moving in the same time on the half of the contour ofthe window 85 from a starting point t₀ placed on the verticalsymmetrical axe of the window 85 to an end point t_(final).

In example 2-8, with reference to FIG. 24, the shape of the window 85 isrepresented by two spatialized sounds S86 f 1, and S86 f 2, thatvirtually moves on the contour of the window from the starting point t₀the end point t_(final), while the spatialized sounds S86 f 1, and S86 f2 being encoded with different frequency, depending on the distancerelative to the user of the contour of the window part 85 travelled bythe spatialized sound.

In example 2-9, with reference to FIG. 25, the shape of the window 85 isrepresented by a single spatialized sound S86 that virtually moves fromthe starting point t₀ on the contour of the window part 85 until itreaches back to the starting point starting point t₀ .

In example 2-10, with reference to FIG. 26, the shape of the window 85is represented by the single spatialized sound S86 that virtually movesin an angled pattern within the space between the interior contour, andexterior contour of the window 85.

In examples 2.11, and 2.12 with reference to FIGS. 27, and 28, inanother preferred embodiments, the selected window can stand for awindow or for a door. In these two examples the window has at least oneinterior frame, either for decorative purpose or with a technicalfunction. The contour of the interior frame is important in thesepreferred embodiments, for example because of the risk of injury orbecause the handle of the door or window is on the interior frame.

In example 2.11 with reference to FIG. 27, there are two windows 85-1,and 85-2 standing together as potential point of interest, placed atdifferent distances relative to the visually impaired user. They can bewindows or doors. The stand together as potential point of interestbecause the action needed to be taken by the visually impaired userconcerns both. For example, there are two separated windows that must beboth open or there is a hallway with two doors through which thevisually impaired user must pass.

The two windows 85-1 and 85-2 are separated by open space of variousdimensions (e.g.: 5-10 cm in case of windows or 1-2 meters in case ofthe doors). The visually impaired user is placed closer to the window851.

The visually impaired user sends the request for the Soundrepresentation of the open space distance between the two windows 85-1and 85-2 as well as for the shape of the interior frame of the twowindows 85-1 and 85-2.

The Sound representation sub-unit 36 extracts the specific informationfrom the Live map 310 of the two windows 85-1 and 85-2, encodes it intospatialized sounds S86 t-1, and S86 t-2 having different timecharacteristics for representing the shape of the interior frames of thetwo windows 85-1 and 85-2.

The window 85-1 placed closer to the visually impaired user, as it isshown in the FIG. 27, is represented by two spatialized sounds S86 t1-1, and S86 t 2-1, one spatialized sound S86 t 1-1 representing theoutside contour of the window 85-1, and the other spatialized sound S86t 2-1 representing internal contour of the window 85-1. Each of thespatialized sounds S86 t 1-1, and S86 t 2-1 comprise two otherspatialized sounds, that start at the same time, S86 t 11-1 with S86 t12-1 representing the outside contour of the window 85-1, and S86 t 21-1with S86 t 21-1 representing internal contour of the window 85-1.

Because of the open space between the two windows 85-1 and 85-2, thewindow 85-1 placed closer to the visually impaired user acts like abarrier for detecting detailed information regarding the second window85-2, consequently the The Sound representation sub-unit 36 is able onlyto output a simplified spatialized sound S86 t-2 corresponding to thethree-dimensional position of the window 85-2 and its verticaldimension.

In example 2-12, with reference to FIG. 28, in another preferredembodiment, the visually impaired user sends the request for the soundrepresentation of the details of the shape of the interior frame of thewindow 85.

For simplicity, FIG. 28 illustrates a rectangular shape, but it can beany geometrical shape. The Sound representation sub-unit 36 extracts thespecific information from the Live map 310, encodes it into twospatialized sounds S861, and S862 starting at the same time, virtuallymoving in an angled pattern within the space between the contour of theinterior frame, and the exterior contour of the window 85.

While the description of the method and the system was disclosed indetail in connection to preferred embodiments, those skilled in the artwill appreciate that modifications may be made to adapt a particularsituation without departing from the essential scope to the teaching ofthe invention.

Reference numerals appearing in the claims are by way of illustrationonly and shall have no limiting effect on the scope of the claims.

The invention claimed is:
 1. A computer-implemented method comprising:acquiring data from an environment of a visually impaired user,comprising a sensory unit of a wearable device sensing from a field ofview, sending the acquired data to a sensory fusion sub-unit of aprocessing and control unit of the wearable device, fusing the acquireddata by the sensory fusion sub-unit, sending the fused data to a livemap sub-unit of the processing and control unit, creating, repeatedlyupdating, and storing, by the live map sub-unit, a live map thatcomprises: one or more live map determinations that are generated basedon the fused data received at the processing and control unit from thesensory fusion sub-unit, including: a position and an orientation of thesensory unit, a plurality of objects, and a plurality of living beings,one or more live map determinations that are generated based on aplurality of relationships between the plurality of objects or theplurality of living beings or between the plurality of objects and theplurality of living beings that are received from a relationship managersub-unit of the processing and control unit, one or more live mapdeterminations that are generated based on a free area that is definedas an ensemble of areas on a ground not occupied by the plurality ofobjects and the plurality of living beings, the free area including: awalkable area that satisfies a set of permanent predetermined walkablearea requirements, and a conditional walkable area that satisfies theset of permanent predetermined walkable area requirements, and at leastone predictable conditional walkable area requirement, automatically orin response to a first request from the visually impaired user,determining, by a navigation manager sub-unit of the processing andcontrol unit, repeatedly updating and storing, at least one navigationpath and associated navigation guiding instructions for the visuallyimpaired user to navigate from a current position of the sensory unit toa point of interest selected among the plurality of objects or theplurality of living beings or the plurality of objects and the pluralityof living beings, automatically or in response to a second request fromthe visually impaired user, repeatedly selecting a preferred navigationpath from the at least one navigation path that (i) passes through thewalkable area or on the conditional walkable area or on the walkablearea and on the conditional walkable area, and (ii) meets a set ofsafety requirements including a non-collision requirement, and anon-aggressivity requirement, wherein any request from the visuallyimpaired user is made by using haptic means or audio means of a usercommands interface the requests being received by the navigation managersub-unit via a user commands interface manager sub-unit of theprocessing and control unit, transmitting, by the navigation managersub-unit to a feedback manager sub-unit of the processing and controlunit, the preferred navigation path and the associated navigationguiding instructions, wherein, when the preferred navigation path passesthrough the conditional walkable area, the navigation manager sub-unitsends to the feedback manager sub-unit the associated navigation guidinginstruction corresponding to the at least one predictable conditionalwalkable area requirement; providing, by the feedback manager sub-unit,guidance to the visually impaired user, along the preferred navigationpath, using guiding modes for transmitting each associated navigationguiding instruction, each navigation instruction comprising haptic orauditory cues sent by the feedback manager sub-unit to a feedback unitof the processing and control unit, the feedback unit comprising: hapticfeedback actuators configured for placement on the head of the visuallyimpaired user, or auditory feedback actuators configured for placementto one or both ears of the visually impaired user, or haptic feedbackactuators configured for placement on the head of the visually impaireduser and auditory feedback actuators configured for placement to one orboth ears of the visually impaired user wherein the guiding modes foreach associated navigation guiding instruction are selected by thevisually impaired user by the user commands interface and through usercommands that are received by the feedback manager sub-unit via the usercommands interface manager sub-unit.
 2. The computer-implemented methodof claim 1, comprising: creating and updating the live map, comprising:repeatedly determining the position and orientation of the sensory unit,a position, orientation and characteristics of the plurality of objectsand of the plurality of living beings, based on the fused data receivedfrom the sensory fusion sub-unit, and repeatedly sending the created andupdated live map to a localisation module of the sensory fusionsub-unit, repeatedly generating and updating, by the relationshipmanager sub-unit, the plurality of relationships between the pluralityof objects or the plurality of living beings or the plurality of objectsand the plurality of living beings based on the data acquired from thelive map comprising: applying a set of the predetermined relationsrequirements, and repeatedly sending the updated plurality ofrelationships to the live map, repeatedly localizing, by a localisationmodule the position and orientation of the sensory unit with respect tothe plurality of the objects, and, to the plurality of living beings ofthe live map using localisation algorithms applied to the data receivedfrom the sensory unit and data from the live map and repeatedly sendingthe localisation data of the position and orientation of the sensoryunit to a walkable area detection module of the sensory fusion sub-unit,repeatedly determining, by the walkable area detection module, the freearea based on: the data received from the sensory unit, the datareceived from the localisation module, the set of permanentpredetermined walkable area requirements, and the at least onepredictable conditional walkable area requirement calculated and storedin the memory, and repeatedly sending the updated free area to the livemap; and repeatedly storing the updated live map in the memory.
 3. Thecomputer-implemented method of claim 1, wherein the live map is updatedby the sensory fusion sub-unit using simultaneous localisation andmapping (SLAM) algorithms.
 4. The computer-implemented method of claim1, comprising, sending an information request by the visually impaireduser to a sound representation sub-unit of the processing and controlunit regarding at least one object selected from the plurality ofobjects or at least one living being selected from the plurality ofliving beings; extracting by a sound representation sub-unit of theprocessing and control unit from the live map the information regardingthe selected at least one particular object or at least one particularliving being; representing the extracted information as correspondingspatialized sounds; transmitting the spatialized sounds to the visuallyimpaired user by the feedback unit; selecting, by the visually impaireduser of the point of interest from the plurality of objects or from theplurality of living beings; and transmitting the corresponding selectionrequest to the navigation manager sub-unit.
 5. The computer-implementedmethod of claim 1, comprising: determining by the navigation managerwandering path together with the associated navigation guidinginstructions for the visually impaired user, and sending the wanderingpath and the associated navigation guiding instructions to the feedbackmanager sub-unit.
 6. The computer-implemented method of claim 1, whereinthe haptic cues vary in duration, periodicity, intensity or frequency ofthe vibration according to predetermined preferred navigation pathcomplexity criteria, and wherein the audio cues vary in frequencies,duration, repetition intensity, or 3d spatial virtualization accordingto the predetermined preferred navigation path complexity criteria. 7.The computer-implemented method of claim 1, wherein a three-dimensionalwalkable tunnel is defined as a virtual tunnel of predeterminedcross-section, having as horizontal longitudinal axis the preferrednavigation path, and wherein the guiding mode further comprises specifichaptic cues sent to the visually impaired user when the visuallyimpaired user is approaching the virtual walls of the walkable tunnel.8. The computer-implemented method of claim 1, wherein the preferrednavigation path is divided into predetermined segments delimited by aplurality of milestones, and wherein the guiding mode comprises hapticcues or auditory cues signalling the position of a next at least onemilestone providing associated navigation guiding instructions to thevisually impaired user from a current milestone to a subsequentmilestone, and wherein the length of the predetermined segments variesdepending on the complexity and length of the preferred navigation path.9. The computer-implemented method of claim 1, wherein the guiding modecomprises haptic cues or auditory cues or haptic and auditory cuessignalling a direction on the preferred navigation path.
 10. Thecomputer-implemented method of claim 9, wherein the direction on thepreferred navigation path is determined by the line defined by an originof the sensory unit and an intersection of the preferred navigation pathwith a circle having an origin at the position of the sensory unit and aradius with a predetermined length, and wherein the auditory cuessignalling the direction on the preferred navigation path originate froma spatialized sound source placed at a predetermined first distance ofthe spatialized sound source s with respect to the sensory unit.
 11. Thecomputer-implemented method of claim 1 wherein the auditory cues arespatialized sounds originating from a spatialized sound source thatvirtually travels along a predetermined second distance on the preferrednavigation path from the position of the sensory unit until thespatialized sound source reaches the end of the predetermined seconddistance and back to the position of the sensory unit.
 12. A systemcomprising: one or more processors; and one or more non-transitorymachine-readable storage devices storing instructions that areexecutable by the one or more processors to perform operationscomprising: acquiring data from an environment of a visually impaireduser, comprising a sensory unit of a wearable device sensing from afield of view, sending the acquired data to a sensory fusion sub-unit ofa processing and control unit of the wearable device, fusing theacquired data by the sensory fusion sub-unit, sending the fused data toa live map sub-unit of the processing and control unit, creating,repeatedly updating, and storing, by the live map sub-unit, a live mapthat comprises: one or more live map determinations that are generatedbased on the fused data received at the processing and control unit fromthe sensory fusion sub-unit, including: a position and an orientation ofthe sensory unit, a plurality of objects, and a plurality of livingbeings, one or more live map determinations that are generated based ona plurality of relationships between the plurality of objects or theplurality of living beings or between the plurality of objects and theplurality of living beings that are received from a relationship managersub-unit of the processing and control unit, one or more live mapdeterminations that are generated based on a free area that is definedas an ensemble of areas on a ground not occupied by the plurality ofobjects and the plurality of living beings, the free area including: awalkable area that satisfies a set of permanent predetermined walkablearea requirements, and a conditional walkable area that satisfies theset of permanent predetermined walkable area requirements, and at leastone predictable conditional walkable area requirement, automatically orin response to a first request from the visually impaired user,determining, by a navigation manager sub-unit of the processing andcontrol unit, repeatedly updating and storing, at least one navigationpath and associated navigation guiding instructions for the visuallyimpaired user to navigate from a current position of the sensory unit toa point of interest selected among the plurality of objects or theplurality of living beings or the plurality of objects and the pluralityof living beings, automatically or in response to a second request fromthe visually impaired user, repeatedly selecting a preferred navigationpath from the at least one navigation path that (i) passes through thewalkable area or on the conditional walkable area or on the walkablearea and on the conditional walkable area, and (ii) meets a set ofsafety requirements including a non-collision requirement, and anon-aggressivity requirement, wherein any request from the visuallyimpaired user is made by using haptic means or audio means of a usercommands interface the requests being received by the navigation managersub-unit via a user commands interface manager sub-unit of theprocessing and control unit, transmitting, by the navigation managersub-unit to a feedback manager sub-unit of the processing and controlunit, the preferred navigation path and the associated navigationguiding instructions, wherein, when the preferred navigation path passesthrough the conditional walkable area, the navigation manager sub-unitsends to the feedback manager sub-unit the associated navigation guidinginstruction corresponding to the at least one predictable conditionalwalkable area requirement; providing, by the feedback manager sub-unit,guidance to the visually impaired user, along the preferred navigationpath, using guiding modes for transmitting each associated navigationguiding instruction, each navigation instruction comprising haptic orauditory cues sent by the feedback manager sub-unit to a feedback unitof the processing and control unit, the feedback unit comprising: hapticfeedback actuators configured for placement on the head of the visuallyimpaired user, or auditory feedback actuators configured for placementto one or both ears of the visually impaired user, or haptic feedbackactuators configured for placement on the head of the visually impaireduser and auditory feedback actuators configured for placement to one orboth ears of the visually impaired user wherein the guiding modes foreach associated navigation guiding instruction are selected by thevisually impaired user by the user commands interface and through usercommands that are received by the feedback manager sub-unit via the usercommands interface manager sub-unit.
 13. The system of claim 12, whereinthe operations comprise: creating and updating the live map, comprising:repeatedly determining the position and orientation of the sensory unit,a position, orientation and characteristics of the plurality of objectsand of the plurality of living beings, based on the fused data receivedfrom the sensory fusion sub-unit, and repeatedly sending the created andupdated live map to a localisation module of the sensory fusionsub-unit, repeatedly generating and updating, by the relationshipmanager sub-unit, the plurality of relationships between the pluralityof objects or the plurality of living beings or the plurality of objectsand the plurality of living beings based on the data acquired from thelive map comprising: applying a set of the predetermined relationsrequirements, and repeatedly sending the updated plurality ofrelationships to the live map, repeatedly localizing, by a localisationmodule the position and orientation of the sensory unit with respect tothe plurality of the objects, and, to the plurality of living beings ofthe live map using localisation algorithms applied to the data receivedfrom the sensory unit and data from the live map and repeatedly sendingthe localisation data of the position and orientation of the sensoryunit to a walkable area detection module of the sensory fusion sub-unit,repeatedly determining, by the walkable area detection module, the freearea based on: the data received from the sensory unit, the datareceived from the localisation module, the set of permanentpredetermined walkable area requirements, and the at least onepredictable conditional walkable area requirement calculated and storedin the memory, and repeatedly sending the updated free area to the livemap; and repeatedly storing the updated live map in the memory.
 14. Thesystem of claim 12, wherein the live map is updated by the sensoryfusion sub-unit using simultaneous localisation and mapping (SLAM)algorithms.
 15. The system of claim 12, wherein the operations comprise,sending an information request by the visually impaired user to a soundrepresentation sub-unit of the processing and control unit regarding atleast one object selected from the plurality of objects or at least oneliving being selected from the plurality of living beings; extracting bya sound representation sub-unit of the processing and control unit fromthe live map the information regarding the selected at least oneparticular object or at least one particular living being; representingthe extracted information as corresponding spatialized sounds;transmitting the spatialized sounds to the visually impaired user by thefeedback unit; selecting, by the visually impaired user of the point ofinterest from the plurality of objects or from the plurality of livingbeings; and transmitting the corresponding selection request to thenavigation manager sub-unit.
 16. The system of claim 12, wherein theoperations comprise: determining by the navigation manager wanderingpath together with the associated navigation guiding instructions forthe visually impaired user, and sending the wandering path and theassociated navigation guiding instructions to the feedback managersub-unit.
 17. The system of claim 12, wherein the haptic cues vary induration, periodicity, intensity or frequency of the vibration accordingto predetermined preferred navigation path complexity criteria, andwherein the audio cues vary in frequencies, duration, repetitionintensity, or 3d spatial virtualization according to the predeterminedpreferred navigation path complexity criteria.
 18. The system of claim12, wherein a three-dimensional walkable tunnel is defined as a virtualtunnel of predetermined cross-section, having as horizontal longitudinalaxis the preferred navigation path, and wherein the guiding mode furthercomprises specific haptic cues sent to the visually impaired user whenthe visually impaired user is approaching the virtual walls of thewalkable tunnel.
 19. A non-transitory computer storage medium encodedwith a computer program, the computer program comprising instructionsthat when executed by one or more processors cause the one or moreprocessors to perform operations comprising: acquiring data from anenvironment of a visually impaired user, comprising a sensory unit of awearable device sensing from a field of view, sending the acquired datato a sensory fusion sub-unit of a processing and control unit of thewearable device, fusing the acquired data by the sensory fusionsub-unit, sending the fused data to a live map sub-unit of theprocessing and control unit, creating, repeatedly updating, and storing,by the live map sub-unit, a live map that comprises: one or more livemap determinations that are generated based on the fused data receivedat the processing and control unit from the sensory fusion sub-unit,including: a position and an orientation of the sensory unit, aplurality of objects, and a plurality of living beings, one or more livemap determinations that are generated based on a plurality ofrelationships between the plurality of objects or the plurality ofliving beings or between the plurality of objects and the plurality ofliving beings that are received from a relationship manager sub-unit ofthe processing and control unit, one or more live map determinationsthat are generated based on a free area that is defined as an ensembleof areas on a ground not occupied by the plurality of objects and theplurality of living beings, the free area including: a walkable areathat satisfies a set of permanent predetermined walkable arearequirements, and a conditional walkable area that satisfies the set ofpermanent predetermined walkable area requirements, and at least onepredictable conditional walkable area requirement, automatically or inresponse to a first request from the visually impaired user,determining, by a navigation manager sub-unit of the processing andcontrol unit, repeatedly updating and storing, at least one navigationpath and associated navigation guiding instructions for the visuallyimpaired user to navigate from a current position of the sensory unit toa point of interest selected among the plurality of objects or theplurality of living beings or the plurality of objects and the pluralityof living beings, automatically or in response to a second request fromthe visually impaired user, repeatedly selecting a preferred navigationpath from the at least one navigation path that (i) passes through thewalkable area or on the conditional walkable area or on the walkablearea and on the conditional walkable area, and (ii) meets a set ofsafety requirements including a non-collision requirement, and anon-aggressivity requirement, wherein any request from the visuallyimpaired user is made by using haptic means or audio means of a usercommands interface the requests being received by the navigation managersub-unit via a user commands interface manager sub-unit of theprocessing and control unit, transmitting, by the navigation managersub-unit to a feedback manager sub-unit of the processing and controlunit, the preferred navigation path and the associated navigationguiding instructions, wherein, when the preferred navigation path passesthrough the conditional walkable area, the navigation manager sub-unitsends to the feedback manager sub-unit the associated navigation guidinginstruction corresponding to the at least one predictable conditionalwalkable area requirement; providing, by the feedback manager sub-unit,guidance to the visually impaired user, along the preferred navigationpath, using guiding modes for transmitting each associated navigationguiding instruction, each navigation instruction comprising haptic orauditory cues sent by the feedback manager sub-unit to a feedback unitof the processing and control unit, the feedback unit comprising: hapticfeedback actuators configured for placement on the head of the visuallyimpaired user, or auditory feedback actuators configured for placementto one or both ears of the visually impaired user, or haptic feedbackactuators configured for placement on the head of the visually impaireduser and auditory feedback actuators configured for placement to one orboth ears of the visually impaired user wherein the guiding modes foreach associated navigation guiding instruction are selected by thevisually impaired user by the user commands interface and through usercommands that are received by the feedback manager sub-unit via the usercommands interface manager sub-unit.